Is the Nx Cloud up? Visit our Status Page for the current health and performance of the Nx Cloud.

Status Page

Analytics API: object track + auto best shot created for every trackId, even with nonIndexable

matteo lacirignola

Hi all,

Me&Claude are working on an external integration via API (JSON-RPC) and we are having some trouble getting the system to work the way we’d like. We’d appreciate your opinion; I apologize in advance if the information below is incorrect and\or not so clear, please ask me to clarify in case.

Goal:
Our edge detects and tracks many objects per camera (~170 new objects per minute). Only a few situations matter, and we send them as analytics events; some are confirmed only seconds or tens of seconds after they start. When an operator opens an event, we want a replay of a few seconds around it with the boxes of the objects involved, drawn from our detections. We don't need the individual objects to be searchable, and we'd rather not flood the Objects tab with them.

What we do now
- We send a box for every tracked object in real time (metadata.object.create), so the boxes are already in place when an event arrives later. Object type is "nonIndexable" (we also tried
 "liveOnly", and "noAutoBestShots" in the engine manifest).

Observed
- Boxes are drawn correctly on live and archive playback.
- The server still creates one object track per trackId: empty objectTypeId, null engine, no
 positions, automatic best shot ("<Unknown track>" in the Objects tab).
- Reusing a small pool of trackIds (e.g. 4 per camera) gives only 4 records, boxes still look fine.
- Boxes sent more than ~4 s after their frame timestamp are not drawn during archive playback
 (4.0 s ok, 4.5 s not), although the track is in the analytics DB.

Questions
1. Is there a recommended approach for this goal (event replay with integration-drawn boxes)?
  We're open to a different strategy.
2. Is an object track with auto best shot per trackId expected for nonIndexable/liveOnly types,
  despite noAutoBestShots? Can it be avoided?
3. Is reusing a small pool of trackIds safe?
4. Are these records deleted with the archive by retention?
5. Is there a fixed time window for object metadata to be shown in archive playback? Configurable?

Thanks for support,

Matteo

0

Comments

3 comments

Date Votes
  • Ichiro
    • Network Optix team

    Hi 

    Thanks for the detailed write-up. Your goal is clear and a common one, and the API/SDK supports it out of the box already. 

    The nonIndexable and liveOnly flags each have their own behavior and purpose. For your case, nonIndexable looks like the right fit. Before you go further with the implementation, please check the taxonomy.md in the SDK. 

    It explains both flags in detail and is the best reference for this. If you're using Claude or another AI coding tool, give it that file as context will be helpful too.

    noAutoBestShots isn't related to this issue, so we will keep it separate in this thread. This flag provides the feature that if the plugin doesn't send a Best Shot explicitly, the Server won't use the first frame of the Object Track as a fallback. The track will just have no Best Shot.

     

    One important point: a trackId should never be reused for different objects.

    Each distinct object needs its own unique trackId. Reusing trackIds can cause problems across the whole integration, so please check that your implementation, including any AI-generated code, never reuses them if they are different objects.

     

    Here are answers to your questions:

    1. Is there a recommended approach for this goal (event replay with integration-drawn boxes)?

    Yes, this is already supported, and nonIndexable is the way to do it. I'd suggest using the standard approach with nonIndexable and a unique trackId per different object. 

    1. Is an object track with an auto best shot per trackId expected for nonIndexable/liveOnly types, despite noAutoBestShots? Can it be avoided?

    No, if you set no bestshot, then it won't create any.
    Your issue may come from the trackId reuse instead of bestshot at all. The rule is one distinct object, one trackId. If two detections are the same object, sharing a trackId is fine. If they're different objects, each needs its own.

    1. Is reusing a small pool of trackIds safe?

    No. Please don't reuse trackIds for different objects. Every new, distinct object should get its own unique trackId. Your edge detection should be able to tell whether it's seeing the same object again. When it is the same object, the detector gives it the same identifier, so you know it's the same object and not a new one.

    A trackId works the same way: it represents one object's track through the scene. It needs to be set correctly and no duplication for different objects, and only the detection side can provide this information, since it's the part that knows which object is which.

    1. Are these records deleted with the archive by retention?

    Not at the moment. Analytics storage has no size limit or matching cleanup mechanism, so the data is kept for as long as the database exists.

    1. Is there a fixed time window for object metadata to be shown in archive playback? Is it configurable?

    No, this isn't supported right now. Metadata is shown for as long as the object is present.



    The trackId, you can see the sample code of the SDK.

     

    - Boxes sent more than ~4 s after their frame timestamp are not drawn during archive playback (4.0 s ok, 4.5 s not), although the track is in the analytics DB.

    The metadata itself is kept. But if it arrives late and the object has already moved past, there's no point in showing it. The Server handles this internally: if a bounding box's timestamp is more than 2 seconds away from the matching video frame, the overlay and bounding box aren't shown.

    So what you're seeing is most likely expected behavior. It usually means the metadata is being matched to the wrong objects, or there's a delay somewhere in your implementation. That's in your code, so you're in the best position to troubleshoot it. A good first step is to compare the timestamps you attach to each bounding box with the frame timestamps they're meant to match, ideally, they should be always matched instead of seconds delay.

     

    Thanks.

    0
  • matteo lacirignola

    Thanks a lot Ichiro, this is very helpful, especially the point about analytics data not being removed by archive retention.

    Two clarifications, because I think part of my report was misread.

    1. trackIds and the "<Unknown track>" records

    Our production code never reuses trackIds. Each physical object gets its own UUID, derived from the device, the timestamp at which the object is first seen and its index in that frame. The small pool of trackIds was only a deliberate experiment, to see whether the number of records depends on the number of trackIds. It does, and we have dropped it.

    The records appear with unique trackIds too. With this setup:
    - Nx Meta 6.1.2.42921, integration via the Analytics API (not a C++ plugin)
    - object type with "flags": "nonIndexable"
    - "capabilities": "noAutoBestShots" in the engine manifest, confirmed in analyticsTaxonomyDescriptors (engineDescriptors.<ours>.capabilities = noAutoBestShots)
    - no bestShot.create sent at all

    the server still creates one object track per trackId, about 140-190 per minute for one camera. Each has an empty objectTypeId, a null engine, startTimeMs == endTimeMs, no positions, and an automatic best shot taken from the secondary stream. The client shows them as "<Unknown track>" in the Objects tab. In a separate test, 24 objects with 24 distinct trackIds produced 24 such records.

    So my question is: is noAutoBestShots honoured for integrations that use the Analytics API? If not, how can an API integration draw boxes on live and archive video without creating these records? Given that analytics data is never deleted, this matters a lot for us.

    2. The ~4 s limit in archive playback

    Our box timestamps match the frame timestamps exactly. We checked this by requesting from Nx the frame at the timestamp we attach to a box: it is the right frame, to the millisecond, in live and in archive. In playback the boxes appear from the first to the last frame they belong to, so the 2-second matching rule is not what we are seeing.

    What we varied in the test was only when each box is sent, with its timestamp always correct:
    - sent up to 4.0 s after its frame: drawn in archive playback
    - sent 4.5 s or later after its frame: not drawn, although it is in the analytics DB

    Is there a fixed server-side buffer (for example the archive/muxer one) that decides this? Is it fixed or configurable? We would like to know how much latency we can safely allow between a frame and its metadata.

    Thanks again,
    Matteo

    0
  • matteo lacirignola

     Follow-up with a clean re-test on 6.1.2.42921 (Analytics API integration, unique trackId per object, no bestShot sent) finished now:

    - type without flags → ~170 typed object tracks/min (expected);
    - nonIndexable → ~160 tracks/min with empty objectTypeId, null engine, startTimeMs == endTimeMs and an automatic best shot from the secondary stream ("<Unknown track>");
    - nonIndexable + "capabilities": "noAutoBestShots" in the engine manifest (shown in engineDescriptors), with a server restart after the manifest was set → identical, ~160/min.
    So noAutoBestShots does not seem to be honoured for Analytics API integrations. Is this a known issue, or is there another way for an API integration to avoid these records, or we are missing something?
    Also, on retention: with archive retention set to 1 day, all object tracks older than the archive were gone from the analytics DB after a server restart. So it seems they are cleaned up with the archive, at least in this setup.

    Thanks again,

    Matteo

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post