Today Nvidia officially launched its most powerful card-sized IoT GPU ever, the Nvidia Jetson Xavier NX (dev kit $399). We covered the basics of the Xavier NX and its industry-leading MLPerf stats when it was announced in November, but since then we’ve had a chance to get our hands on an early version of the device and dev kit and do some real work on them. Along with the dev kit, Nvidia also introduced cloud-native deployment for Jetson using docker containers, which we also had a chance to try out.
Nvidia Jetson Xavier NX by the Numbers
Built on its Volta architecture, the Jetson Xavier NX is a massive performance upgrade compared with the TX2 and becomes a bigger-sibling to the Jetson Nano. It features 384 CUDA cores, 48 Tensor cores, and 2 Nvidia Deep Learning Accelerators (DLA) engines. Nvidia rates it for 21 Trillion Operations per Second (TOPS) for deep learning performance. Along with the GPU is a reasonably-capable 6-core Nvidia Carmel ARM 64-bit CPU with 6MB of L2 and 4MB of L3 cache. The processor also includes 8GB of 128-bit LPDDR4x RAM with 51.8GB/s bandwidth.
All that fits in a module the size of a credit card that consumes 15 watts — or 10 watts in a power-limited mode. As with earlier Jetson products, the Xavier NX runs Nvidia’s deep-learning software stack, including advanced analytic systems like DeepStream. For connectivity, the developer kit version includes a microSD slot for the OS and applications, as well as 2 MIPI camera connectors, Gigabit Ethernet, M.2 Key E with Wi-Fi/Bluetooth, and an open M.2 Key M for an optional NVMe SSD. Both an HDMI and DisplayPort connector are provided, along with 4 USB 3.1 and 1 USB 2 micro-USB port.
Cloud-Native Deployment Thanks to Docker Containers
It’s one thing to come up with a great industrial or service robot product, but another to keep it up to date and competitive over time. As new technologies emerge, or requirements evolve, update and software maintenance are a major issue. With Xavier NX, Nvidia is also launching its “cloud native” architecture as an option for deploying embedded systems. Now, I’m not personally a fan of slapping “cloud-native” onto technologies just because it is a buzzword. But in this case, at least the benefits of the underlying feature set are clear.
Basically, individual applications and services can be packaged as Docker containers and individually distributed and updated via the cloud. Nvidia sent us a pre-configured SSD loaded with demos, but I was also able to successfully re-format it and download all the relevant Docker containers with just a few commands, which was pretty slick.
Putting the Xavier NX Through Its Paces
Nvidia put together an impressive set of demos as part of the Xavier NX review units. The most sophisticated of them loads a set of docker containers that demonstrate the variety of applications that might be running on an advanced service robot. That includes recognizing people in four HD camera streams, doing full-body pose detection for nearby people in another stream, gaze detection for someone facing the robot, and natural language processing using one of the BERT family of models and a custom corpus of topics and answers.
Nvidia took pains to point out that the demo models have not been optimized for either performance or memory requirements, but aside from requiring some additional SSD space, they still all ran fairly seamlessly on a Xavier NX that I’d set to 15-watt / 6-core mode. To help mimic a real workday, I left the demo running for 8 hours and the system didn’t overheat or crash. Very impressive for a credit-card-sized GPU!
Running multiple Docker container-based demos on the Nvidia Jetson Xavier NX.
The demo uses canned videos, as otherwise, it’d be very hard to recreate in a review. But based on my experience with its smaller sibling, the Jetson Nano, it should be pretty easy to replicate with a combination of directly-attached camera modules, USB cameras, and cameras streaming over the internet. Third-party support during the review period is pretty tricky, as the product was still under NDA. I’m hoping that once it is out I’ll be able to attach a RealSense camera that reports depth along with video, and perhaps write a demo app that shows how far apart the people in a scene are from each other.
Developing for the Jetson Xavier NX
Being ExtremeTech, we had to push past the demos for some coding. Fortunately, I had just the project. I foolishly agreed to help my colleague Joel with his magnum opus project of creating better renderings of various Star Trek series. My task was to come up with an AI-based video upscaler that we could train on known good and poor versions of some episodes and then use it to re-render the others. So in parallel to getting on setup on my desktop using my Nvidia 1080, I decided to see what would happen if I worked on the Xavier NX.
Nvidia makes development — especially video and AI development — deceptively easy on its Jetson devices. Its JetPack toolset comes with a lot of AI frameworks pre-loaded, and Nvidia’s excellent developer support sites offer downloadable packages for many others. There is also plenty of tutorial content for local development, remote development, and cross-compiling. The deceptive bit is that you get so comfortable that you just about forget that you’re developing on an ARM CPU.
At least until you stumble across a library or module that only runs on x86. That happened to me with my first choice of super-resolution frameworks, an advanced GAN-based approach, mmsr. Mmsr itself is written in Python, which is always encouraging as far as being cross-platform, but it relies on a tricked-out deformation module that I couldn’t get to build on the Jetson. I backed off to an older, simpler, CNN-based scaler, SRCNN, which I was able to get running. Training speed was only a fraction of my 1080, but that’s to be expected. Once I get everything working, the Xavier NX should be a great solution for actually grinding away on the inference-based task of doing the scaling.
Is a Xavier NX Coming to a Robot Near You?
In short, probably. To put it in perspective, the highly-capable Skydio autonomous drone uses the older TX2 board to navigate obstacles and follow subjects in real time. The Xavier NX provides many times (around 10x in pure TOPS numbers) the performance in an even smaller form factor. It’s also a great option for DIY home video applications or hobby robot projects.
Now Read:
- Nvidia’s new Jetson Xavier NX Adds Horsepower to AI at the Edge
- Hands On With Nvidia’s New Jetson Nano
- Hands On With Nvidia’s JetBot AI-Powered DIY Robot
No comments:
Post a Comment