Month: April 2019

Uncategorized

FRC 2019 – Camera Best Practices

Posted on by Keith Rieck

To get the most out of your cameras for the FRC 2019, please consider following these recommendations. This document does not contain the theory for the recommendations. If the theory is desired or for any questions regarding these recommendations, please contact a MN CSA at firstmn.csa@gmail.com or http://firstmncsa.slack.com.

Desired goals that drive these recommendations

  • Low latency
    • Allows driver to react to the most current status with a minimal delay between driver input and robot action cycle time.
  • Low bandwidth usage
    • Reduced risk of driver input being delayed due to high bandwidth.
      • There is a Quality of Service mechanism that should prevent this, but to fully eliminate the risk, reduce bandwidth if possible.
    • Bandwidth target is below 3/Mbs
  • Ease of use

Possible Misconceptions

  • Higher FPS means lower latency.
    • While higher FPS can appear to reduce latency in a video game, that only occurs when the underlying infrastructure can support the desired FPS with minimal latency to begin with.
    • Low latency is a function of the infrastructure’s ability to get data from point a, the camera, to point b, the DS screen, with minimal delays. This can only occur if that infrastructure has available waiting capacity to process, transmit and display the video.
    • Higher FPS can easily overload the underlying infrastructure, which can cause delays at every stage of the point a to point b pipeline, thus increasing the overall latency.
    • Lowering FPS to a level which the infrastructure can handle the pipeline while still maintaining available waiting capacity, will assist in achieving the lowest possible latency.
  • High Resolution is better
    • High resolution is desirable if additional detail allows for a strategic advantage, but for most tasks, lower latency will offer a much better robot control experience.
    • 640×480 is not twice as much as 320×240. It is 4 times as much. The extra time required to process, transmit and display 4 times the data is most likely not going to offset the higher latency and reduce capacity required for its use.
  • This or that device is the right one for all tasks.
    • Not all devices work well in all situations, you should balance the total cost to implement, maintain and configure additional devices before making changes. Cost in this sense means monetary, time, expertise, weight, etc…

Driver Cam

  • Use FRCVision on Raspberry PI instead of cameras hosted on roboRIO
  • URL: https://wpilib.screenstepslive.com/s/currentCS/m/85074/l/1027241-using-the-raspberry-pi-for-frc
  • Benefits:
    • Potential for robot code to respond faster to driver input by offloading CPU intensive task from roboRIO.
    • Lower video latency and higher frame rates due to increased cpu cycles available on Pi.
    • Ability to handle more concurrent streams than a roboRIO.
    • Ability to control stream from FRC shuffleboard and LabView Dashboard.
    • Ability to control Resolution, FPS and compression per camera feed.
    • Ability to have a per camera vision processing pipeline.
    • Multiple language choices for vision processing pipeline.
    • No need to add code for basic camera streaming.
  • Recommended Usage:
    • Driver video streaming.
    • Video processing, target acquisition and tracking.
  • Recommended Equipment:
    • Raspberry Pi 3 B or B+, B+ preferred.
    • Microsoft Lifecam HD-3000
    • Logitech c920, c930, c270, c310
    • Any Linux UVC  supported USB camera that supports MJPEG and desired resolution and fps in camera hardware: http://www.ideasonboard.org/uvc/#devices
  • Optional Equipment:
  • Recommended hardware settings, per camera.
    • Format: MJPEG
    • Resolution: 320×240
    • FPS: 15-20, reduce as needed to reduce Pi cpu usage.
  • Recommended stream settings, per camera
    • Format: MJPEG
    • Resolution: 320×240
    • FPS: 10-15, reduce as needed to lower Pi cpu usage or bandwidth
    • Compression: 30, adjust as needed to get desired cpu usage, bandwidth and clarity.
  • Considerations:
    • Power:, ensure 2.5+ amps of power are available to the Pi, especially if using 3-4 cameras and / or vision processing pipeline is in use.
    • Actual FPS per video stream as listed in the DS view should match set FPS for the stream as configured for that camera, if it does not, lower FPS and / or Resolution or increase compression until actual FPS and set FPS match and video quality and latency is acceptable.
          • Rather than using driver mode, create a “driver” pipeline. Turn down the exposure to reduce stream bandwidth.
          • Using a USB camera? Use the “stream” NT key to enable picture-in-picture mode. This will dramatically reduce stream bandwidth.
          • Turn the stream rate to “low” in the settings page if streaming isn’t critical for driving.
        • Considerations:
          • Do NOT use for driver vision.
          • Use only for target acquisition and tracking.
          • Stream only output of vision pipeline to DS and only if bandwidth allows.

Chris Roadfeldt

0