classifyImage

warning

The document is a continuation of the previous document, if you have landed directly on this page then, Please read from page Get started.

What is classifyImage component ?

classifyImage component is used to classify image.

• Description : classifyImage() takes an input as image in base64 format and N i.e. top N probability of classes with probabilities. Check Input and output parameters for details.
• Parameters :
  • Input(Via STDIN) : classifyImage.stdin() << jsonDoc2Text(inputJson) << std::endl;
    • An inputJson String with following parameters:
      • Parameter1 : Image (.jpg/.png) in base64 format
      • Parameter2 : N : Top N class predictions (value between 1 to 1000)
  • Output(Via STDOUT) : classifyImage.stdout() >> outputJson;
    • An outputJson string with following contents
      • For each class in top n
        • Score/Probabillity
        • Index
      • int requestID

List of classifyImage features in shunya stack

1. Set N, get top N predictions
2. Get class data.

Using classifyImage

Requirements to use classifyImage

1. Shunya OS installed (supported Arm devices) or Shunya OS docker container (X86 based windows/linux devices)
2. Shunya AI installed in Shunya OS.
3. Shunya Video component installed in Shunya OS.

Steps to use classifyImage

1. Set input video location.
2. Read Video frames
3. Convert cv image to base64 string
4. Customize top N prediction value.
5. Print classified image's output

note

Run the steps given below inside Shunya OS installed (supported Arm devices) or Shunya OS docker container (X86 based windows/linux devices) terminals.

Lets take an example use case: Say we need to

1. Get top N classes predictions
2. Get JSON output of classified image.

Steps are

Step 1: Set input video location

1. Start with a ready to use template for classfication from video/image.

   git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git

2. Open the examples in a text editor and modify as per your usecase.
   • We will use video compnent to set input video location in configuration file.
   • Please check here, how to setup the video source path in video component.

Step 2: Read video frames

1. Once video path is set using video component, lets read video frames one by one.

   • For CPP you will find the examples in the folder cpp-examples/image-classification/
   • Open the file image_classify.cpp
   • Code to read first video frame. If you want to read all frames one by one, put following code in while loop.

   /* --- Capturing image from video using Video component --- */
   captureObj src = newCaptureDevice("camera"); /* Create capture Instance */
   cv::Mat inputImage;
   int32_t outIndex = 0;
   
   /*################## Call Video Component functions ################*/
   inputImage  = captureFrameToMem(&src); /* Capture one frame at a time in a loop*/
   
   if (inputImage.empty()) {
       fprintf(stderr, "End of video file!.");
       closeCapture(&src);
       return 0;
   }

Step 3: Convert cv image to base64 string

1. Since classifyImage API needs image in base64 string format, we will convert image from cv::Mat to base64 string.
2. Code to do it

   /* ---- Create cv::mat image to base64 string ---- */
   std::string b64InpImg = mat2Base64(inputImage);

Step 4: Customize top N prediction value.

1. Open the examples in a text editor and modify as per your usecase.

   • For CPP you will find the examples in the folder cpp-examples/image-classification/classifyImage
   • Open the file classify_image.cpp
   • Modify the line to set the N.

   /* Replace 2 with your N value */
   int nprob = 2;

Step 5: Call API binary

1. We will now call API binary by giving input image(parameter1) and N value(parameter2) an input through STDIN.

   /* Creating inputJson adding image and N value in it */
   rapidjson::Document inputJson;
   inputJson.SetObject();
   rapidjson::Value inpImage;
   
   inpImage.SetString(b64InpImg.c_str(), strlen(b64InpImg.c_str()), inputJson.GetAllocator());
   /* Adding parameter1 in inputJson file */
   inputJson.AddMember("inputImage", inpImage, inputJson.GetAllocator());
   /* Adding parameter2 in inputJson file */
   inputJson.AddMember("nprob", nprob, inputJson.GetAllocator());
   /*################ Call classifyImage Component #####################*/
   subprocess::popen classifyImage("/usr/bin/classifyImage", {});
   /* Calling classifyImage by passing inputJson as an input through STDIN */
   classifyImage.stdin() << jsonDoc2Text(inputJson) << std::endl;
   classifyImage.close();
   std::string outputJson;
   /* Getting output in outputJson */
   classifyImage.stdout() >> outputJson;

2. You will get output in outputJson string.

Step 6: Print output.

1. Code to print the json output, got from classifyImage API.

   std::cout<<outputJson<<"\n\n";

Run ready to use example.

1. Once you are done editing, save and run the code, by running

   git clone https://gitlab.iotiot.in/shunya/products/shunya-ai-examples.git
   cd cpp-examples/image-classification/
   mkdir build && cd build
   cmake ../
   make 
   ./imageClassifyCpp

2. Running the codes will print the JSON output on the terminal (to STDOUT).

   For Example:

   • Lets say the input image is

     

   • Input JSON is

     {
         "inputImage": "4h32e898473urmcrkd947ryuemcc3x21j98k09754ycmx1k030i1d98754yn==",
         "nprob": 2
     }

   • Then the JSON output is

     {
         "apiVersion": "1.2.0",
         "requestId": 19287918271287,
         "data": {
             "classes": [
                 {
                     "class": "Cat",
                     "confidence": 99.97504425048828         
                 },
                 {
                     "class": "Dog",
                     "confidence": 99.97504425048828         
                 },
             ],
         }
     }

Related Components to classifyImage that you may need to complete your application

• no related components

Facing errors with the component?