-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Testing BytedecoVideoDataLogger to see if it is compatible * Setup to use Bytedeco video capture and print timestamps * Moved files to Robot Data Logger for bytedeco video capture * Testing to ensure that all VideoCapture work correctly * Switch to the capture ID from Logger * Disabled video preview * Debugging with the logger * Added flags and messed around with capture ID * Added methods to grab frames using different classes. * Testing different grabbers * Test grabbing methods * Added functionality for Gstreamer * Added Renderer Class and fixed saving to file * Updated GStreamer pipeline to save a video as an mp4 that has moov data for the MOV container. Vidoe doesn't get saved properly but this is the correct pipeline. * Wroking GStreamer that records video which can be either an mov or an mp4. Runs for 10 seconds with no frame capture. * Added tracker for frames, it will now print something when a frame should be saved. * Saves a video with the correct timesstamps and evertying is working correctly for real on security camera * Code cleanup * Renameing * Renaming * Plays GStreamer videos from actual Logger classes * Code comments, cleanup * Added deploy file, and print statement so we know gstreamer is running * Added camera index * Change file to run on logger cameras * Notes on how to fix timestamps * Changing timestamps to work with SCS2Visualizer * Added note to look into new generated error * Changing how the timestamps get mapped and need to test on robot. * Timestamps with GStreamer but now the buffer isn't writable, need to go back to personal machine to look at it. * Changed timestamp print statments * Fixed the example * Changed frames per second to 60 * Updated capture example to reflect more of what the logger is actually doing with timestamps * Ready for testing on Logger with HDMI plugin, currently SDI doesn't work and its pretty weird. Doesn't work with SCS2Visualizer without robotdata so testing on that needs to be with robot * Should work with Robot running, works with logger just doesn't generate the robotTimeStamps * Works on robot with robottimestamp, playing back in SCS2Visualizer could be better I think * Working on Valkyrie tag, timestamps and SCS2Visualizer playback is good. CURRENTLY DOES NOT WORK WITH VIDEODATAPLAYER * If a variable is static it won't be an instance of a class, but part of the main method running so there can only be one of the variables in existance. * Removed print statments for BlackmagicVideoDataLogger * Update notes * Whole bunch of stuff, got the windows example for Bytedeco to work well. I'm able to play it back with Bytedeco in windows as well. Small other changes. Lots of testing * Newer version of Bytedeco.javaCV and cleaned up some files and more doc to make the better * Bumped library version * Setup recorders to work with MageWell capture device * Renamed, cleaned up comments to attempt to make things more readable from the terminal * Got the capture type working and updated pubsub with the new message types * Cleanup of MagewellVideoDataLogger and somehow added stuff to the camera class? Need to see if we need that comment * Updated examples and added better comments * Removed the ability to just export a video, wasn't being used, seperated out a MagewellMuxer and MagewellDemuxer to enable cropping logs corrrectly * Just a complete mess, need to go to other desk and sort it with my multiple monitors * Updated the way we crop logs with MagewellMuxer, cropping logs now works * Check that the next frame is greater then the previous one * Cleaned up tests and implementation, added docs * Bug fixes with video times stamps, video fps, video quality * Change method to be pulled into a tools class to only be written in one place, added helpful comments --------- Co-authored-by: OperatorComputer <OperatorComputer@unknown> Co-authored-by: nkitchel <nkitchel@ihmc.us>
- Loading branch information
1 parent
5f6919b
commit 426167a
Showing
15 changed files
with
600 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
210 changes: 210 additions & 0 deletions
210
src/main/java/us/ihmc/robotDataLogger/logger/GStreamerVideoDataLogger.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,210 @@ | ||
package us.ihmc.robotDataLogger.logger; | ||
|
||
import java.io.File; | ||
import java.io.FileWriter; | ||
import java.io.IOException; | ||
import java.util.concurrent.Semaphore; | ||
|
||
import org.freedesktop.gstreamer.*; | ||
import org.freedesktop.gstreamer.event.EOSEvent; | ||
import us.ihmc.commons.Conversions; | ||
import us.ihmc.javadecklink.CaptureHandler; | ||
import us.ihmc.log.LogTools; | ||
import us.ihmc.robotDataLogger.LogProperties; | ||
import us.ihmc.tools.maps.CircularLongMap; | ||
|
||
/** | ||
* This class does not work properly yet, it still needs more testing to make sure everything works as expected | ||
* In order to prevent usage without more testing this class is deprecated since it's still got bugs in it | ||
*/ | ||
@Deprecated | ||
public class GStreamerVideoDataLogger extends VideoDataLoggerInterface implements CaptureHandler | ||
{ | ||
/** | ||
* Make sure to set a progressive mode, otherwise the timestamps will be all wrong! | ||
*/ | ||
|
||
private static boolean WRITTEN_TO_TIMESTAMP = false; | ||
private static long latestHardwareTimestamp; | ||
|
||
private FileWriter timestampWriter; | ||
private final Semaphore gotEOSPlayBin = new Semaphore(1); | ||
private final CircularLongMap circularLongMap = new CircularLongMap(10000); | ||
private Pipeline pipeline; | ||
|
||
private int frameNumber; | ||
private final int decklinkID; | ||
private static volatile long lastFrameTimestamp = 0; | ||
private static int i = 0; | ||
|
||
|
||
public GStreamerVideoDataLogger(String name, File logPath, LogProperties logProperties, int decklinkID, YoVariableLoggerOptions options) throws IOException | ||
{ | ||
super(logPath, null, logProperties, name); | ||
|
||
this.decklinkID = decklinkID; | ||
createCaptureInterface(); | ||
} | ||
|
||
private void createCaptureInterface() throws IOException | ||
{ | ||
File timestampFile = new File(timestampData); | ||
File videoCaptureFile = new File(videoFile); | ||
|
||
timestampWriter = new FileWriter(timestampFile); | ||
|
||
startCapture(videoCaptureFile); | ||
} | ||
|
||
public void startCapture(File videoCaptureFie) | ||
{ | ||
LogTools.info("Starting Gstreamer with camera index: " + decklinkID); | ||
Gst.init(); | ||
|
||
String deckLinkIndex = " device-number=" + decklinkID + " "; | ||
|
||
pipeline = (Pipeline) Gst.parseLaunch( | ||
"decklinkvideosrc connection=hdmi " + deckLinkIndex + | ||
"! timeoverlay " + | ||
"! videorate ! video/x-raw,framerate=60/1 " + | ||
"! identity name=identity " + | ||
"! jpegenc " + | ||
"! .video splitmuxsink muxer=qtmux location=" + videoCaptureFie); | ||
|
||
pipeline.getBus().connect((Bus.EOS) (source) -> | ||
{ | ||
System.out.println("Recieved the EOS on the pipeline!"); | ||
gotEOSPlayBin.release(); | ||
}); | ||
|
||
gotEOSPlayBin.drainPermits(); | ||
|
||
Element identity = pipeline.getElementByName("identity"); | ||
identity.getStaticPad("sink") | ||
.addProbe(PadProbeType.BUFFER, new TimestampProbe()); | ||
|
||
pipeline.play(); | ||
} | ||
|
||
/* | ||
* (non-Javadoc) | ||
* @see us.ihmc.robotDataLogger.logger.VideoDataLoggerInterface#restart() | ||
*/ | ||
@Override | ||
public void restart() throws IOException | ||
{ | ||
close(); | ||
removeLogFiles(); | ||
createCaptureInterface(); | ||
} | ||
|
||
/* | ||
* (non-Javadoc) | ||
* @see us.ihmc.robotDataLogger.logger.VideoDataLoggerInterface#timestampChanged(long) | ||
*/ | ||
@Override | ||
public void timestampChanged(long newTimestamp) | ||
{ | ||
if (pipeline != null) | ||
{ | ||
if (latestHardwareTimestamp != System.currentTimeMillis()) | ||
{ | ||
circularLongMap.insert(System.currentTimeMillis(), newTimestamp); | ||
latestHardwareTimestamp = System.currentTimeMillis(); | ||
} | ||
} | ||
} | ||
|
||
/* | ||
* (non-Javadoc) | ||
* @see us.ihmc.robotDataLogger.logger.VideoDataLoggerInterface#close() | ||
*/ | ||
@Override | ||
public void close() | ||
{ | ||
LogTools.info("Signalling recorder to shut down."); | ||
try | ||
{ | ||
LogTools.info("Stopping capture."); | ||
pipeline.sendEvent(new EOSEvent()); | ||
gotEOSPlayBin.acquire(1); | ||
pipeline.stop(); | ||
|
||
LogTools.info("Closing writer."); | ||
timestampWriter.close(); | ||
|
||
LogTools.info("Done."); | ||
} | ||
catch (IOException | InterruptedException e) | ||
{ | ||
e.printStackTrace(); | ||
} | ||
|
||
timestampWriter = null; | ||
} | ||
|
||
@Override | ||
public void receivedFrameAtTime(long hardwareTime, long pts, long timeScaleNumerator, long timeScaleDenumerator) | ||
{ | ||
|
||
if (circularLongMap.size() > 0) | ||
{ | ||
long robotTimestamp = circularLongMap.getValue(true, hardwareTime); | ||
|
||
if (frameNumber % 420 == 0) | ||
{ | ||
double delayInSeconds = Conversions.nanosecondsToSeconds(circularLongMap.getLatestKey() - hardwareTime); | ||
System.out.println("Delay in Seconds: " + delayInSeconds + " / PresentationTimeStamp: " + pts); | ||
} | ||
|
||
try | ||
{ | ||
if (!WRITTEN_TO_TIMESTAMP) | ||
{ | ||
timestampWriter.write(timeScaleNumerator + "\n"); | ||
timestampWriter.write(timeScaleDenumerator + "\n"); | ||
WRITTEN_TO_TIMESTAMP = true; | ||
} | ||
|
||
timestampWriter.write(robotTimestamp + " " + pts + "\n"); | ||
|
||
lastFrameTimestamp = System.nanoTime(); | ||
} | ||
catch (IOException e) | ||
{ | ||
e.printStackTrace(); | ||
} | ||
|
||
++frameNumber; | ||
} | ||
} | ||
|
||
@Override | ||
public long getLastFrameReceivedTimestamp() | ||
{ | ||
return lastFrameTimestamp; | ||
} | ||
|
||
|
||
class TimestampProbe implements Pad.PROBE | ||
{ | ||
@Override | ||
public PadProbeReturn probeCallback(Pad pad, PadProbeInfo info) | ||
{ | ||
Buffer buffer = info.getBuffer(); | ||
|
||
if (buffer.isWritable()) | ||
{ | ||
//One will work with SCS2Visualizer, the other works with VideoDataPlayer | ||
// receivedFrameAtTime(System.currentTimeMillis(), buffer.getPresentationTimestamp(), 1, 60000); | ||
receivedFrameAtTime(System.currentTimeMillis(), i, 1, 90000); | ||
// Val setup Timestamps | ||
i += 3001; | ||
// Nothing setup Timestamps | ||
i += 1001; | ||
} | ||
|
||
return PadProbeReturn.OK; | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.