Jetson Inference
DNN Vision Library
|
Hardware-accelerated video encoder and decoder (H.264/H.265) using GStreamer. More...
Classes | |
class | gstBufferManager |
gstBufferManager recieves GStreamer buffers from appsink elements and unpacks/maps them into CUDA address space, and handles colorspace conversion into RGB format. More... | |
class | gstDecoder |
Hardware-accelerated video decoder for Jetson using GStreamer. More... | |
class | gstEncoder |
Hardware-accelerated video encoder for Jetson using GStreamer. More... | |
class | gstWebRTC |
Static class for common WebRTC utility functions used with GStreamer. More... | |
Macros | |
#define | LOG_GSTREAMER "[gstreamer] " |
LOG_GSTREAMER logging prefix. More... | |
Hardware-accelerated video encoder and decoder (H.264/H.265) using GStreamer.
class gstBufferManager |
gstBufferManager recieves GStreamer buffers from appsink elements and unpacks/maps them into CUDA address space, and handles colorspace conversion into RGB format.
It can handle both normal CPU-based GStreamer buffers and NVMM memory which can be mapped directly to the GPU without requiring memory copies using the CPU.
To disable the use of NVMM memory, set -DENABLE_NVMM=OFF when building with CMake:
cmake -DENABLE_NVMM=OFF ../
Public Member Functions | |
gstBufferManager (videoOptions *options) | |
Constructor. More... | |
~gstBufferManager () | |
Destructor. More... | |
bool | Enqueue (GstBuffer *buffer, GstCaps *caps) |
Enqueue a GstBuffer from GStreamer. More... | |
int | Dequeue (void **output, imageFormat format, uint64_t timeout=UINT64_MAX) |
Dequeue the next frame. More... | |
uint64_t | GetLastTimestamp () const |
Get timestamp of the latest dequeued frame. More... | |
imageFormat | GetRawFormat () const |
Get raw image format. More... | |
uint64_t | GetFrameCount () const |
Get the total number of frames that have been recieved. More... | |
Protected Attributes | |
imageFormat | mFormatYUV |
The YUV colorspace format coming from appsink (typically NV12 or YUY2) More... | |
RingBuffer | mBufferYUV |
Ringbuffer of CPU-based YUV frames (non-NVMM) that come from appsink. More... | |
RingBuffer | mTimestamps |
Ringbuffer of timestamps that come from appsink. More... | |
RingBuffer | mBufferRGB |
Ringbuffer of frames that have been converted to RGB colorspace. More... | |
uint64_t | mLastTimestamp |
Timestamp of the latest dequeued frame. More... | |
Event | mWaitEvent |
Event that gets triggered when a new frame is recieved. More... | |
videoOptions * | mOptions |
Options of the gstDecoder / gstCamera object. More... | |
uint64_t | mFrameCount |
Total number of frames that have been recieved. More... | |
bool | mNvmmUsed |
Is NVMM memory actually used by the stream? More... | |
gstBufferManager::gstBufferManager | ( | videoOptions * | options | ) |
Constructor.
gstBufferManager::~gstBufferManager | ( | ) |
Destructor.
int gstBufferManager::Dequeue | ( | void ** | output, |
imageFormat | format, | ||
uint64_t | timeout = UINT64_MAX |
||
) |
Dequeue the next frame.
Returns 1 on success, 0 on timeout, -1 on error.
bool gstBufferManager::Enqueue | ( | GstBuffer * | buffer, |
GstCaps * | caps | ||
) |
Enqueue a GstBuffer from GStreamer.
|
inline |
Get the total number of frames that have been recieved.
|
inline |
Get timestamp of the latest dequeued frame.
|
inline |
Get raw image format.
|
protected |
Ringbuffer of frames that have been converted to RGB colorspace.
|
protected |
Ringbuffer of CPU-based YUV frames (non-NVMM) that come from appsink.
|
protected |
The YUV colorspace format coming from appsink (typically NV12 or YUY2)
|
protected |
Total number of frames that have been recieved.
|
protected |
Timestamp of the latest dequeued frame.
|
protected |
Is NVMM memory actually used by the stream?
|
protected |
Options of the gstDecoder / gstCamera object.
|
protected |
Ringbuffer of timestamps that come from appsink.
class gstDecoder |
Hardware-accelerated video decoder for Jetson using GStreamer.
gstDecoder supports loading video files from disk (MKV, MP4, AVI, FLV) and RTP/RTSP network streams over UDP/IP. The supported decoder codecs are H.264, H.265, VP8, VP9, MPEG-2, MPEG-4, and MJPEG.
Public Member Functions | |
~gstDecoder () | |
Destructor. More... | |
virtual bool | Capture (void **image, imageFormat format, uint64_t timeout=DEFAULT_TIMEOUT, int *status=NULL) |
Capture the next decoded frame. More... | |
virtual bool | Open () |
Open the stream. More... | |
virtual void | Close () |
Close the stream. More... | |
bool | IsEOS () const |
Return true if End Of Stream (EOS) has been reached. More... | |
virtual uint32_t | GetType () const |
Return the interface type (gstDecoder::Type) More... | |
Public Member Functions inherited from videoSource | |
virtual | ~videoSource () |
Destroy interface and release all resources. More... | |
template<typename T > | |
bool | Capture (T **image, int *status) |
Capture the next image from the video stream, using the default timeout of 1000ms. More... | |
template<typename T > | |
bool | Capture (T **image, uint64_t timeout=DEFAULT_TIMEOUT, int *status=NULL) |
Capture the next image from the video stream. More... | |
bool | IsStreaming () const |
Check if the device is actively streaming or not. More... | |
uint32_t | GetWidth () const |
Return the width of the stream, in pixels. More... | |
uint32_t | GetHeight () const |
Return the height of the stream, in pixels. More... | |
uint32_t | GetFrameRate () const |
Return the framerate, in Hz or FPS. More... | |
uint64_t | GetFrameCount () const |
Return the number of frames captured. More... | |
uint64_t | GetLastTimestamp () const |
Get timestamp of the last captured frame, in nanoseconds. More... | |
imageFormat | GetRawFormat () const |
Get raw image format. More... | |
const URI & | GetResource () const |
Return the resource URI of the stream. More... | |
const videoOptions & | GetOptions () const |
Return the videoOptions of the stream. More... | |
bool | IsType (uint32_t type) const |
Check if this stream is of a particular type. More... | |
template<typename T > | |
bool | IsType () const |
Check if a this stream is of a particular type. More... | |
const char * | TypeToStr () const |
Convert this stream's class type to string. More... | |
Static Public Member Functions | |
static gstDecoder * | Create (const videoOptions &options) |
Create a decoder from the provided video options. More... | |
static gstDecoder * | Create (const URI &resource, videoOptions::Codec codec) |
Create a decoder instance from resource URI and codec. More... | |
static bool | IsSupportedExtension (const char *ext) |
Return true if the extension is in the list of SupportedExtensions. More... | |
Static Public Member Functions inherited from videoSource | |
static videoSource * | Create (const videoOptions &options) |
Create videoSource interface from a videoOptions struct that's already been filled out. More... | |
static videoSource * | Create (const char *URI, const videoOptions &options=videoOptions()) |
Create videoSource interface from a resource URI string and optional videoOptions. More... | |
static videoSource * | Create (const char *URI, const commandLine &cmdLine) |
Create videoSource interface from a resource URI string and parsing command line arguments. More... | |
static videoSource * | Create (const char *URI, const int argc, char **argv) |
Create videoSource interface from a resource URI string and parsing command line arguments. More... | |
static videoSource * | Create (const int argc, char **argv, int positionArg=-1) |
Create videoSource interface by parsing command line arguments, including the resource URI. More... | |
static videoSource * | Create (const commandLine &cmdLine, int positionArg=-1) |
Create videoSource interface by parsing command line arguments, including the resource URI. More... | |
static const char * | Usage () |
Usage string for command line arguments to Create() More... | |
static const char * | TypeToStr (uint32_t type) |
Convert a class type to a string. More... | |
Static Public Attributes | |
static const uint32_t | Type = (1 << 1) |
Unique type identifier of gstDecoder class. More... | |
static const char * | SupportedExtensions [] |
String array of supported video file extensions, terminated with a NULL sentinel value. More... | |
Static Public Attributes inherited from videoSource | |
static const uint64_t | DEFAULT_TIMEOUT =1000 |
The default Capture timeout (1000ms) More... | |
Protected Member Functions | |
gstDecoder (const videoOptions &options) | |
void | checkMsgBus () |
void | checkBuffer () |
bool | buildLaunchStr () |
bool | discover () |
bool | init () |
bool | initPipeline () |
void | destroyPipeline () |
bool | isLooping () const |
Protected Member Functions inherited from videoSource | |
videoSource (const videoOptions &options) | |
Static Protected Member Functions | |
static void | onEOS (_GstAppSink *sink, void *user_data) |
static GstFlowReturn | onPreroll (_GstAppSink *sink, void *user_data) |
static GstFlowReturn | onBuffer (_GstAppSink *sink, void *user_data) |
static void | onWebsocketMessage (WebRTCPeer *peer, const char *message, size_t message_size, void *user_data) |
Protected Attributes | |
GstBus * | mBus |
GstElement * | mPipeline |
_GstAppSink * | mAppSink |
Event | mWaitEvent |
std::string | mLaunchStr |
bool | mCustomSize |
bool | mCustomRate |
bool | mEOS |
size_t | mLoopCount |
gstBufferManager * | mBufferManager |
WebRTCServer * | mWebRTCServer |
bool | mWebRTCConnected |
Protected Attributes inherited from videoSource | |
bool | mStreaming |
videoOptions | mOptions |
uint64_t | mLastTimestamp |
imageFormat | mRawFormat |
Additional Inherited Members | |
Public Types inherited from videoSource | |
enum | Status { ERROR = -2, EOS = -1, TIMEOUT = 0, OK = 1 } |
Stream status codes that are optionally returned from Capture() More... | |
gstDecoder::~gstDecoder | ( | ) |
Destructor.
|
protected |
|
protected |
|
virtual |
|
protected |
|
protected |
|
virtual |
|
static |
Create a decoder instance from resource URI and codec.
|
static |
Create a decoder from the provided video options.
|
protected |
|
protected |
|
inlinevirtual |
Return the interface type (gstDecoder::Type)
Reimplemented from videoSource.
|
protected |
|
protected |
|
inline |
Return true if End Of Stream (EOS) has been reached.
In the context of gstDecoder, EOS means that playback has reached the end of the file, and looping is either disabled or all loops have already been run. In the case of RTP/RTSP, it means that the stream has terminated.
|
inlineprotected |
|
static |
Return true if the extension is in the list of SupportedExtensions.
ext | string containing the extension to be checked (should not contain leading dot) |
|
staticprotected |
|
staticprotected |
|
staticprotected |
|
staticprotected |
|
virtual |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
static |
String array of supported video file extensions, terminated with a NULL sentinel value.
The supported extension are:
|
static |
Unique type identifier of gstDecoder class.
class gstEncoder |
Hardware-accelerated video encoder for Jetson using GStreamer.
The encoder can write the encoded video to disk in (MKV, MP4, AVI, FLV), or stream over the network to a remote host via RTP/RTSP using UDP/IP. The supported encoder codecs are H.264, H.265, VP8, VP9, and MJPEG.
Public Member Functions | |
~gstEncoder () | |
Destructor. More... | |
template<typename T > | |
bool | Render (T *image, uint32_t width, uint32_t height) |
Encode the next frame. More... | |
virtual bool | Render (void *image, uint32_t width, uint32_t height, imageFormat format) |
Encode the next frame. More... | |
virtual bool | Open () |
Open the stream. More... | |
virtual void | Close () |
Close the stream. More... | |
GstPipeline * | GetPipeline () const |
Return the GStreamer pipeline object. More... | |
WebRTCServer * | GetWebRTCServer () const |
Return the WebRTC server (only used when the protocol is "webrtc://") More... | |
virtual uint32_t | GetType () const |
Return the interface type (gstEncoder::Type) More... | |
Public Member Functions inherited from videoOutput | |
virtual | ~videoOutput () |
Destroy interface and release all resources. More... | |
template<typename T > | |
bool | Render (T *image, uint32_t width, uint32_t height) |
Render and output the next frame to the stream. More... | |
bool | IsStreaming () const |
Check if the device is actively streaming or not. More... | |
uint32_t | GetWidth () const |
Return the width of the stream, in pixels. More... | |
uint32_t | GetHeight () const |
Return the height of the stream, in pixels. More... | |
float | GetFrameRate () const |
Return the framerate, in Hz or FPS. More... | |
uint64_t | GetFrameCount () const |
Return the number of frames output. More... | |
const URI & | GetResource () const |
Return the resource URI of the stream. More... | |
const videoOptions & | GetOptions () const |
Return the videoOptions of the stream. More... | |
void | AddOutput (videoOutput *output) |
Add an output sub-stream. More... | |
uint32_t | GetNumOutputs () const |
Return the number of sub-streams. More... | |
videoOutput * | GetOutput (uint32_t index) const |
Return a sub-stream. More... | |
virtual void | SetStatus (const char *str) |
Set a status string (i.e. More... | |
bool | IsType (uint32_t type) const |
Check if this stream is of a particular type. More... | |
template<typename T > | |
bool | IsType () const |
Check if a this stream is of a particular type. More... | |
const char * | TypeToStr () const |
Convert this stream's class type to string. More... | |
Static Public Member Functions | |
static gstEncoder * | Create (const videoOptions &options) |
Create an encoder from the provided video options. More... | |
static gstEncoder * | Create (const URI &resource, videoOptions::Codec codec) |
Create an encoder instance from resource URI and codec. More... | |
static bool | IsSupportedExtension (const char *ext) |
Return true if the extension is in the list of SupportedExtensions. More... | |
Static Public Member Functions inherited from videoOutput | |
static videoOutput * | Create (const videoOptions &options) |
Create videoOutput interface from a videoOptions struct that's already been filled out. More... | |
static videoOutput * | Create (const char *URI, const videoOptions &options=videoOptions()) |
Create videoOutput interface from a resource URI string and optional videoOptions. More... | |
static videoOutput * | Create (const char *URI, const commandLine &cmdLine) |
Create videoOutput interface from a resource URI string and parsing command line arguments. More... | |
static videoOutput * | Create (const char *URI, const int argc, char **argv) |
Create videoOutput interface from a resource URI string and parsing command line arguments. More... | |
static videoOutput * | Create (const int argc, char **argv, int positionArg=-1) |
Create videoOutput interface by parsing command line arguments, including the resource URI. More... | |
static videoOutput * | Create (const commandLine &cmdLine, int positionArg=-1) |
Create videoOutput interface by parsing command line arguments, including the resource URI. More... | |
static videoOutput * | CreateNullOutput () |
Create videoOutput interface that acts as a NULL output and does nothing with incoming frames. More... | |
static const char * | Usage () |
Usage string for command line arguments to Create() More... | |
static const char * | TypeToStr (uint32_t type) |
Convert a class type to a string. More... | |
Static Public Attributes | |
static const uint32_t | Type = (1 << 2) |
Unique type identifier of gstEncoder class. More... | |
static const char * | SupportedExtensions [] |
String array of supported video file extensions, terminated with a NULL sentinel value. More... | |
Protected Member Functions | |
gstEncoder (const videoOptions &options) | |
bool | init () |
bool | initPipeline () |
void | destroyPipeline () |
void | checkMsgBus () |
bool | buildCapsStr () |
bool | buildLaunchStr () |
bool | encodeYUV (void *buffer, size_t size) |
Protected Member Functions inherited from videoOutput | |
videoOutput (const videoOptions &options) | |
Static Protected Member Functions | |
static void | onNeedData (GstElement *pipeline, uint32_t size, void *user_data) |
static void | onEnoughData (GstElement *pipeline, void *user_data) |
static void | onWebsocketMessage (WebRTCPeer *peer, const char *message, size_t message_size, void *user_data) |
Protected Attributes | |
GstBus * | mBus |
GstCaps * | mBufferCaps |
GstElement * | mAppSrc |
GstElement * | mPipeline |
bool | mNeedData |
std::string | mCapsStr |
std::string | mLaunchStr |
RingBuffer | mBufferYUV |
RTSPServer * | mRTSPServer |
WebRTCServer * | mWebRTCServer |
Protected Attributes inherited from videoOutput | |
bool | mStreaming |
videoOptions | mOptions |
std::vector< videoOutput * > | mOutputs |
gstEncoder::~gstEncoder | ( | ) |
Destructor.
|
protected |
|
protected |
|
protected |
|
protected |
|
virtual |
|
static |
Create an encoder instance from resource URI and codec.
|
static |
Create an encoder from the provided video options.
|
protected |
|
protected |
|
inline |
Return the GStreamer pipeline object.
|
inlinevirtual |
Return the interface type (gstEncoder::Type)
Reimplemented from videoOutput.
|
inline |
Return the WebRTC server (only used when the protocol is "webrtc://")
|
protected |
|
protected |
|
static |
Return true if the extension is in the list of SupportedExtensions.
ext | string containing the extension to be checked (should not contain leading dot) |
|
staticprotected |
|
staticprotected |
|
staticprotected |
|
virtual |
|
inline |
Encode the next frame.
|
virtual |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
static |
String array of supported video file extensions, terminated with a NULL sentinel value.
The supported extension are:
|
static |
Unique type identifier of gstEncoder class.
class gstWebRTC |
Static class for common WebRTC utility functions used with GStreamer.
This gets used internally by gstEncoder/gstDecoder for handling WebRTC streams.
Static Public Member Functions | |
static void | onNegotiationNeeded (GstElement *webrtcbin, void *user_data) |
Callback for handling webrtcbin "on-negotation-needed" signal. More... | |
static void | onCreateOffer (GstPromise *promise, void *user_data) |
Callback for handling webrtcbin "create-offer" signal. More... | |
static void | onIceCandidate (GstElement *webrtcbin, uint32_t mline_index, char *candidate, void *user_data) |
Callback for handling webrtcbin "on-ice-candidate" signal. More... | |
static void | onWebsocketMessage (WebRTCPeer *peer, const char *message, size_t message_size, void *user_data) |
Handle incoming websocket messages from the client. More... | |
|
static |
Callback for handling webrtcbin "create-offer" signal.
This sends an SDP offer to the client.
|
static |
Callback for handling webrtcbin "on-ice-candidate" signal.
This send an ICE candidate to the client.
|
static |
Callback for handling webrtcbin "on-negotation-needed" signal.
It's expected that user_data is set to a WebRTCPeer instance.
|
static |
Handle incoming websocket messages from the client.
This only handles SDP/ICE messages - it's expected that the caller will handle new peer connecting/closing messages.
#define LOG_GSTREAMER "[gstreamer] " |
LOG_GSTREAMER logging prefix.