We often get questions from customers and prospectives on what use-cases work well with Seervision, or some unexpected failure modes. In this article, the goal is to walk you through how Seervision works behind the scenes, and what that means for your use case.

When we started out with Seervision, our goal was to make a system that was as flexible as possible. We wanted to avoid intrusive hardware when reasonable, while maintaining the flexibility of a human operator. Using Computer Vision is a great way to achieve that goal. Our Computer Vision only needs a single cue: does the subject look human? Regardless of the angle, whether we see a face or not, if the subject of interest “looks” human, we can track them.

How is the Computer Vision implemented? Without going into too much of the gory technical details, here is a rough overview:

1. We take the live video signal, and convert it down to a lower resolution and framerate, so that we can fit more images per second into the GPU (where our analysis runs).
2. We analyse the image frame and detect any humans that may be present
3. If we’re tracking, we locate the VIP, and compare to where the user set the reference point
4. We send commands to the PTU to update its position if necessary
5. Repeat from step 1. This happens multiple times per second.

Of course, Computer Vision isn’t a silver bullet for tracking (not yet anyways!), and there are a few limitations that are important to understand.

• Seervision does not “recognise” or “remember” talents by default. Customers often ask if they can be remembered, in case they walk out of the video feed and back into it. For GDPR reasons, Seervision does not do this by default. You can enable the ‘person specific automation’ (beta) option whereby you can store a person’s likeness or simply upload a photo. In doing so, we can remember talents and always associate the same ID to a specific talent. This way you can track specific people or only, enable person-specific workflows.
• So how do we distinguish talents? Seervision uses aggregate clues – think of facial features, hair color, person size, color of their clothes, etc. These cues help us distinguish talents from one another. This also means that if talents all wear the same uniform, distinguishing them becomes more complex.
• Seervision is not suitable for extremely wide shots. In these shots, the talent that is to be tracked will often be smaller than approximately 1/3rd of the frame (in height). That means that our Computer Vision has very few pixels or ‘visually identifying information’ to go on. In other words, when a person is that small in the frame, there isn’t enough visual information available for us to differentiate between people robustly or do consistent, smooth tracking.
• Seervision detects up to a maximum of 8 people in the frame. This isn’t actually true, we detect every single human-like object in the frame, but you’ll notice that from approximately 8 talents onwards, the system can start to slow down, as it has to analyse the state of each available talent. Note that with some smart usage of the Tracking Zone / Exclusion Zone (see our manual), you can exclude areas from the shot to reduce the load on the system.
• Visually noisy environments can be problematic. With “visually noisy”, we mean situations like excessively flashing lights, smoke, noisy backgrounds, or a combination of all, often found on stages for music festivals for example. While our algorithms have grown robust, these kinds of effects make VIPs look temporarily “different”, impeding tracking.
• Seervision is not suitable for sports. As an extension of everything above, most sports are both too fast, have too many crossings between players (especially in the case of team sports), and all talents “look the same” (same uniform, size, behaviour).

Knowing these limitations, how can you make sure you’re successful when running a Seervision-powered system? We’ve seen quite a few successful deployments at this stage, and we’ve seen the same pattern each time: start out simple with the recommended use-cases, and as you grow more comfortable, explore further out. Concretely:

• Make your first few deployments with Seervision centered around straightforward presentation use-cases, such as keynotes, lectures, and conferences. This will allow you to get comfortable with the features and get to know the system’s robustness.
• As you grow more confident, start experimenting and see what Seervision can enable for your specific usage scenario!

This page has been divided into four sections:

1. Debugging a PTZ setup
2. Debugging a Robotic Head setup
3. Interface-related debugging
4. Getting Support

It may be faster to just Ctrl + F (or Cmd + F if you’re on a Mac) to find key words related to your issue. That’s why we put it all in one page!

Are you getting a video signal?

Make sure you have a video connection from the PTZ to the Seervision server. In most cases, Seervision servers can take at most a FullHD signal. Anything more could result in no image appearing at all.

The only exception to this is if you explicitly opted for a 4k-capable Seervision server.

On the PTZ, make sure you are using the corresponding video output port.

Can you control the PTZ?

If you are moving the PTZ via the UI, but nothing is happening, it’s time to verify the connection between the Seervision server and the PTZ, see below.

Have you correctly configured the PTZ?

Double-check that you’ve correctly modified the settings as specified in the PTZ Quickstart (pan/tilt mode, focus control etc).

Are you getting an SDI/NDI signal?

Make sure you have an SDI connection from the PTU to the Seervision server. In most cases, Seervision servers can take a 3G SDI (FullHD) signal. Anything more will result in no image appearing at all.

The only exception to this is if you explicitly opted for a 4k-capable Seervision Server.

Are you able to control the head?

In the interface, try to move the head, either with the joystick or with the pan/tilt bars at the bottom. If nothing happens:

• Verify that the head is powered up
• Verify that the head is correctly connected to the same LAN network as the Seervision server.
  • If possible, verify using the ping network utility that you can ping both the Seervision server and the robotic head.
• Go into the Operations Server on the Seervision server and verify that the server has got the correct configuration for this head configured.

Are you able to control the lens’ zoom, focus and optionally iris?

If not, there may be an issue with the lens control configuration.

• If you are using external lens motors (like PDMovie ones):

  • Are the lens motors powered on (LEDs should light up) and connected to the head or to the edge device?
  • Are the lens motors calibrated? Try to move the zoom, focus and iris rings by hand. If you can’t, the PDMovies are calbrated. If they move freely, long press on the bottom of the lens motor, which will cause it to automatically calibrate.
  • Are the lens motors LED colors corresponding to their function? Green for focus, red for zoom, blue for iris. To change the color, double press the button on the lens motor.

• If you are using a lens with a servo:

  • Verify that the controls at the bottom set to M (for manual) and not S (for servo)
  • Verify that the servo is connected to the power port on the camera
  • Verify that the servo is connected to the serial control port on the head

Tracking performance is poor

This can be caused by quite a few things and it’s probably best to call in Seervision Support. However, in 80% of the cases, it means the correct lens isn’t selected. Verify that you have configured the correct lens by going to the Preferences (little cogwheel icon) in the UI, navigating to the Lenses tab, and confirming that you have selected both the right camera, as well as the right lens.

If that’s not the case, enable the Computer Vision stream under the Streams panel, to get a better feeling for whether the computer vision is causing the poor performance.

Restarting

As frustrating as it can be to admit, sometimes a restart from scratch can help in fixing issues. We have three options for a restart, ranging from mild to “nuclear”:

In the System panel, click the Reset all Settings button to revert to our default configuration.

If the above didn’t work, head on over to the Operations Server, and click the Relaunch button to relaunch the UI. Allow a few minutes for the system to relaunch.

If that also didn’t work, the “nuclear” option is to just use the Reboot button in the Operations Server, which will restart the Seervision server. This will take a few minutes.

Getting Support

It may happen that the problem is nearly impossible to debug without some “insider information” Seervision Support can provide, no problem!

NOTE: While it can be a frustrating experience, try to provide as much detail as the situation allows when contacting Support. This helps us provide you with more targeted support, and hopefully a quicker resolution time.