Screen Activity

Enable this model configuration to captures the visual activity in a conversation

Overview

Identify and analyze the visual aspects of the meeting along with the corresponding timestamps and with Screen Activity API.

Types

Activity	Description
Speaker/Interaction	when the meeting/conversation has no visual elements or if it is face-to-face (video is ON) conversation.
Sketching/whiteboarding	detects when there is whiteboarding or sketching in the meeting
Presentation	detects full-screen presentations during the meeting
Screen Share	detects general screen share like browsing through the computer, webpages, etc. during a screen share session.
Speaker Face	Detect when the speakers face is displayed in the meeting.

modelTypeConfiguration

Keys	Value
modelType	screen_activity
modelConfig	Model Configuration object for screen_activity

Example Request

Curl

curl --location --request POST 'https://api.marsview.ai/cb/v1/conversation/compute' \
--header 'Content-Type: application/json' \
--header "authorization: {{Insert Auth Token}}" \
--data-raw '{
        "txnId": "{{Insert txn ID}}",
        "enableModels":[
            "modelType":"screengrabs",
            "modelConfig":{
                "ocr":{
                    "enable":true
                    },
                "screen_activity" : {
                    	"enable" : true,
                    	"classes" : ["drawing","ppt","screen',"speaker_face","speaker_presentation"]
                        }
                }
            },
            {
            "modelType":"screen_activity"
            }
        ]
}'

Python

import requests
auth_token = "replace this with your auth token"
txn_id = "Replace this with your txn id"
request_url = "https://api.marsview.ai/cb/v1/conversation/compute"


def get_screen_activity():
  payload={
        "txnId": txn_id,
        "enableModels":[
            {
            "modelType":"screengrabs",
            "modelConfig":{
                "ocr":{
                    "enable":True
                    }
                }
            },
            {
            "modelType":"screen_activity"
            }
            ]
        }
  headers = {'authorization': '{}'.format(auth_token)}
  
  response = requests.request("POST",headers=headers, json=payload)
  print(response.text)
  if response.status_code == 200 and response.json()["status"] == "true":
    return response.json()["data"]["enableModels"]["state"]["status"]
  else:
    raise Exception("Custom exception")

if __name__ == "__main__": 
    get_screen_activity()

Example Response

"data": {
    "meetingActivity": [
    {
        "shotId": 1,
        "startFrame": 1,
        "endFrame": 3237,
        "meetingActivity": [
            {
                "className": "speaker_fused"
            }
        ],
        "meetingActivityAccuracy": {
            "speakerFace": "3.0225227"
        },
        "startTime": 66.66666666666667,
        "endTime": 215800
    }
}

Response Object

Field	Description
meetingActivity	List of meeting activity objects
shotId	Id of the collection of frames being analyzed
startFrame	First frame of the specified shotId
endFrame	Last frame of the specified shotId
meetingActivity[meetingActivity]	Type of meeting activity identified in the shotId
className	Type of class best describing the type of screen activity
meetingActivityAccuracy	Object speicying the accuracy of the predicted meeting activity
speakerFace
startTime	Start time of the shotId in milliseconds
endTime	Start time of the shotId in milliseconds