detectObjects

warning

The document is a continuation of the previous document, if you have landed directly on this page then, Please read from page Get started.

What is detectObjects component ?

detectObjects component is used to detect objects from an input image.

Description : detectObjects() takes an input as image in base64 string format and probability and returns the object information with each object probability greater than input given. Check Input and output parameters for details.
Parameters :
- Input(Via STDIN) : detectObjects.stdin() << jsonDoc2Text(inputJson) << std::endl;
  - An inputJson String with following parameters:
    - Parameter1 : Image (.jpg/.png) in base64 format
    - Parameter2 : Float probability (value between 0 to 1)
- Output(Via STDOUT) : detectObjects.stdout() >> outputJson;
  - An outputJson string with following contents
    - For each object
      - Bounding box co-ordinates
      - Probability being object
      - Class label
    - int requestID

List of detectObjects features in shunya stack

Customize object probability.
Get object data.
Currently we support 20 classes. List below

"aeroplane", "bicycle", "bird", "boat",
"bottle", "bus", "car", "cat", "chair",
"cow", "diningtable", "dog", "horse",
"motorbike", "person", "pottedplant",
"sheep", "sofa", "train", "tvmonitor"

Using `detectObjects`

Requirements to use `detectObjects`

Shunya OS installed (supported Arm devices) or Shunya OS docker container (X86 based windows/linux devices)
Shunya AI installed in Shunya OS.
Shunya Video component installed in Shunya OS.

Steps to use `detectObjects`

Set input video location.
Read Video frames
Convert cv image to base64 string
Customize object probability.
Call API binary
Print detected objects output

note

Run the steps given below inside Shunya OS installed (supported Arm devices) or Shunya OS docker container (X86 based windows/linux devices) terminals.

Lets take an example use case: Say we need to

Detect all those objects which have a probability of 80% or higher in an image.
Get JSON output of all the detected objects.

Steps are

Step 1: Set input video location

Start with a ready to use template for detecting objects from video/image.
```
git clone https://gitlab.iotiot.in/repo-public/examples.git
```
Open the examples in a text editor and modify as per your usecase.
- We will use video compnent to set input video location in configuration file.
- Please check here, how to setup the video source path in video component.

Step 2: Read video frames

Once video path is set using video component, lets read video frames one by one.

For CPP you will find the examples in the folder shunya-ai-examples/indiv-components/cpp-examples/object-detection/detectObjects
Open the file object_detect.cpp
Code to read first video frame. If you want to read all frames one by one, put following code in while loop.

/* --- Capturing image from video using Video component --- */
captureObj src = newCaptureDevice("video-source"); /* Create capture Instance */
cv::Mat inputImage;
int32_t outIndex = 0;

/*################## Call Video Component functions ################*/
inputImage  = captureFrameToMem(&src); /* Capture one frame at a time in a loop*/

if (inputImage.empty()) {
    fprintf(stderr, "End of video file!.");
    closeCapture(&src);
    return 0;
}

Step 3: Convert cv image to base64 string

Since detectObjects API needs image in base64 string format, we will convert image from cv::Mat to base64 string.

Code to do it

/* ---- Create cv::mat image to base64 string ---- */
std::string b64InpImg = mat2Base64(inputImage);

Step 4: Customize object probability

Modify the line to set the object probability to 80% or higher.

/* Set value to 0.8 i.e (80/100) to set the probability to 80% or higher */
float probability = 0.8;  

Step 5: Call API binary

We will now call API binary by giving input image(parameter1) and object probability(parameter2) as an input through STDIN.

/* Creating inputJson adding image and object probability in it */
rapidjson::Document inputJson;
inputJson.SetObject();
rapidjson::Value inpImage;

/* Call API binary */
inpImage.SetString(b64InpImg.c_str(), strlen(b64InpImg.c_str()), inputJson.GetAllocator());
/* Adding parameter1 in inputJson file */
inputJson.AddMember("inputImage", inpImage, inputJson.GetAllocator());
/* Adding parameter2 in inputJson file */
inputJson.AddMember("probability", probability, inputJson.GetAllocator());
/*############### Call detectObjects Component ##################*/
subprocess::popen detectObjects("/usr/bin/detectObjects", {});
/* Calling detectObjects by passing inputJson as an input through STDIN */
detectObjects.stdin() << jsonDoc2Text(inputJson) << std::endl;
detectObjects.close();
std::string outputJson;
/* Getting output in outputJson */
detectObjects.stdout() >> outputJson;

You will get output in outputJson string.

Step 6: Print detected objects output.

Code to print the json output, got from detectObjects API.

/* ---- Printing detected objects information ---- */
rapidjson::Document detectObjectsJson = readJsonString(detectObjectsOut);
if (detectObjectsJson.HasMember("data")) {
   rapidjson::Value &results = detectObjectsJson["data"]["objects"];
   assert(results.IsArray());
   /* Checking if objects are detected or not */
   if(results.Size()>0){
   /* Reading from json file and add it in the structure */
        for (rapidjson::SizeType i = 0; i < results.Size(); i++) {
            // Printing detected objects values
            std::cout<<"\nData:";
            std::cout<<"\nObject"<<std::to_string(i)<<": ";
            std::cout<<"\nObject Label: "<<results[i]["object"].GetString();
            std::cout<<"\nObject probability: "<<results[i]["confidence"].GetFloat();
        }
    }
    else{
        std::cout<<"\nNo objects detected.";
    }
}
else{
        std::cout<<"\nno json member as data ";
}

Run ready to use example.

Once you are done editing, save and run the code, by running

git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git
cd cpp-examples/object-detection/
mkdir build && cd build
cmake ../
make 
./detectObjectsCpp

Running the codes will print the JSON output on the terminal (to STDOUT).

For Example:

Lets say the input image is

Input JSON is

{
    "inputImage": "4h32e898473urmcrkd947ryuemcc3x21j98k09754ycmx1k030i1d98754yn==",
    "probability": 0.8
}

Then the JSON output is

{
    "apiVersion": "1.2.0",
    "requestId": 19287918271287,
    "data": {
        "objects": [
            {
                "object": "Cat", 
                "boundingBox": {
                    "top": 234253,
                    "left": 124.023480,
                    "width": 2435,
                    "height": 2345,
                },
                "confidence": 99.97504425048828         
            },
            {
                "object": "Dog", 
                "boundingBox": {
                    "top": 234253,
                    "left": 124.023480,
                    "width": 2435,
                    "height": 2345,
                },
                "confidence": 99.97504425048828         
            },
        ],
        "image": "base64str"
    }
}

This part is not updated yet (please do not use below example)

Understand this component with an example (ready to use code)

This is an example for object-detection and here we will be using 2 components: detectObjects and drawObjects
Check this ready to use example in c++ and python
Download the code
git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git cd shunya-ai-examples/cpp-examples/object-detection
```shell git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git cd shunya-ai-examples/python-examples/object-detection ```
In this folder there is a file, object_detect.cpp or object_detect.py
detectbjects Components used
subprocess::popen detectObjects("/usr/bin/detectObjects", {});
```shell detectObjects = Popen(['/usr/bin/detectObjects'], stdout=PIPE, stdin=PIPE) ```
drawObjects component used
subprocess::popen drawObjects("/usr/bin/drawObjects", {});
```shell drawObjects = Popen(['/usr/bin/drawObjects'], stdout=PIPE, stdin=PIPE) ```
Run code by yourself
mkdir build && cd build cmake .. && make ./objectDetectCpp
```shell python3 object_detect.py ```
- You will get a new image stored in system.

warning

What is detectObjects component ?​

List of detectObjects features in shunya stack​

Using detectObjects​

Requirements to use detectObjects​

Steps to use detectObjects​

note

Step 1: Set input video location​

Step 2: Read video frames​

Step 3: Convert cv image to base64 string​

Step 4: Customize object probability​

Step 5: Call API binary​

Step 6: Print detected objects output.​

Run ready to use example.​

This part is not updated yet (please do not use below example)

Understand this component with an example (ready to use code)​

Facing errors with the component?​

What is detectObjects component ?

List of detectObjects features in shunya stack

Using `detectObjects`

Requirements to use `detectObjects`

Steps to use `detectObjects`

Step 1: Set input video location

Step 2: Read video frames

Step 3: Convert cv image to base64 string

Step 4: Customize object probability

Step 5: Call API binary

Step 6: Print detected objects output.

Run ready to use example.

Understand this component with an example (ready to use code)

Facing errors with the component?