API configuration between Python Post-Processor and Nx Meta

Hi,

I'm working on a Python post-processor for detecting stationary vehicles at stop lines or the roadside. My Python program looks something like this:

app.py :

from flask import Flask, render_template, request, jsonify, send_file
import numpy as np
import cv2
import time
from io import BytesIO
from PIL import Image

app = Flask(__name__)

# Define the spots with their coordinates
spots = {
    'Spot1': [(865, 326), (1053, 328), (1266, 511), (981, 516)]
}

occupancy = {spot: False for spot in spots}
occupancy_frame_counts = {spot: 0 for spot in spots}
occupancy_timestamps = {spot: time.time() for spot in spots}
occupancy_durations = {spot: 0 for spot in spots}
frame_threshold = 0.35
occupancy_threshold = 0.35

# Track frame counts and midpoints
frame_count = 0

def check_occupancy(bboxes):
    global occupancy, occupancy_frame_counts, occupancy_timestamps, occupancy_durations
    inside_spot = {spot: False for spot in spots.keys()}

    for i in range(0, len(bboxes), 4):
        x_min, y_min, x_max, y_max = map(int, bboxes[i:i+4])

        # Calculate midpoint
        midpoint = ((x_min + x_max) // 2, (y_min + y_max) // 2)

        # Check if midpoint is inside any spot polygon
        for spot, polygon in spots.items():
            mask = np.zeros((1280, 720), dtype=np.uint8)  # Updated resolution to (1280, 720)
            cv2.fillPoly(mask, [np.array(polygon, dtype=np.int32)], color=(255, 255, 255))
            if mask[midpoint[1], midpoint[0]] == 255:
                inside_spot[spot] = True

    # Update occupancy based on inside_spot with frame count logic
    for spot in spots.keys():
        if inside_spot[spot]:
            occupancy_frame_counts[spot] += 1
        else:
            occupancy_frame_counts[spot] -= 1

        # Ensure frame counts stay within [0, frame_threshold]
        occupancy_frame_counts[spot] = min(max(occupancy_frame_counts[spot], 0), frame_threshold)

        # Update the actual occupancy status based on the threshold
        if occupancy_frame_counts[spot] >= occupancy_threshold and not occupancy[spot]:
            occupancy[spot] = True
            occupancy_durations[spot] = time.time() - occupancy_timestamps[spot]
            occupancy_timestamps[spot] = time.time()
        elif occupancy_frame_counts[spot] < occupancy_threshold and occupancy[spot]:
            occupancy[spot] = False
            occupancy_durations[spot] = time.time() - occupancy_timestamps[spot]
            occupancy_timestamps[spot] = time.time()

@app.route('/')
def index():
    return render_template('index3.html', occupancy=occupancy, occupancy_durations=occupancy_durations)

@app.route('/add_message', methods=['POST'])
def add_message():
    global occupancy
    message = request.json
    for key, value in message['BBoxes_xyxy'].items():
        if key != 'ROI':
            check_occupancy(value)
    return '', 204

@app.route('/occupancy', methods=['GET'])
def get_occupancy():
    return jsonify({
        'occupancy': occupancy,
        'empty_spots': sum(1 for occupied in occupancy.values() if not occupied),
        'filled_spots': sum(1 for occupied in occupancy.values() if occupied),
        'occupancy_durations': occupancy_durations
    })

if __name__ == '__main__':
    app.run(debug=True)

The post-processor is simply a Python socket using Flask. For the NX Meta interface, the display looks like this:

I'm confused about how to do the following:

How can I display the ROI (Region of Interest) defined in app.py on the camera feed in the NX Meta interface?
How can I ensure that when a vehicle enters the ROI, the data (e.g., detection or event details) is displayed on a web page?

I tried adding an API in the "Camera Rules" and also tried plugins like "Stub" and "HTTPS request," but the data is still not showing up.

Any guidance would be greatly appreciated.

Ichiro

November 19, 2024 05:39

Hi Fandra Andito,

It seems there may be a few misunderstandings regarding the plugin subsystem, integrations, and data flow in Nx Meta. I’d like to clarify these points for you:

1. Interacting with the Nx Meta Interface:

To display or interact with any specific application, it is necessary to send or push the required data to the application. Currently, your Python script is running independently and only receiving data from other sources— it is not interacting with Nx Meta or its components. As a result, you won’t see anything on the Nx Meta interface from your current implementation.

If your goal is to interact with the desktop client and create new entries in the event rules, it is mandatory to use the MetaSDK to develop a plugin. The Python script, being an external interface or webpage, does not interact directly with the Nx Media Server. That being said, unfortunately, your current implementation cannot fulfill this request.

2. Displaying ROI (Region of Interest) in Nx Meta Interface:

To display an ROI defined in app.py on the Nx Meta camera feed, you must use the MetaSDK to relay your Python script’s configuration to the Nx Witness Media Server. The ROI implementation must be done using MetaSDK in C++, and you can refer to the sample code in the STUB/ROI example.

Note that the Nx Witness ROI can only be applied to device-level video sources (e.g., cameras, RTSP streams) and cannot be displayed on web interfaces. This limitation is inherent to the platform.

3. Displaying Data on a Web Page When a Vehicle Enters the ROI:

If you want to display data (e.g., detection or event details) on a webpage when a vehicle enters the ROI, you’ll need to:

•Create Analytics ROI: Use the MetaSDK to define the ROI (refer to STUB/ROI) and redirect the coordinates from Nx to your webpage. This would involve sending the data externally via HTTP requests within the plugin code.

•Enable Event Rules Integration: Create the manifest and configure the necessary settings to generate and create the analytics events. Refer to the STUB/Events and the sample plugin for guidance.

These functionalities must be implemented using MetaSDK in C++, which may involve a higher development cost. A step-by-step guide on creating analytics events from scratch is available in the public knowledge base.

4. Available Tools and Resources:

While we understand this may not be the most convenient solution, it is currently the best option available. Starting from Nx v6.1.0, a set of Analytics APIs is included in the MetaSDK to simplify such tasks. If you are willing to try the beta version, it is available on the Developer’s Portal.

We strongly recommend reviewing the MetaSDK documentation in the knowledge base before beginning development. These resources provide an essential understanding of the plugin subsystem, including common design concepts, data formats, and configurations.

API configuration between Python Post-Processor and Nx Meta

Comments

Didn't find what you were looking for?