Screen Activity
Enable this model configuration to captures the visual activity in a conversation

Overview

Identify and analyze the visual aspects of the meeting along with the corresponding timestamps and with Screen Activity API.

Types

Activity
Description
Speaker/Interaction
when the meeting/conversation has no visual elements or if it is face-to-face (video is ON) conversation.
Sketching/whiteboarding
detects when there is whiteboarding or sketching in the meeting
Presentation
detects full-screen presentations during the meeting
Screen Share
detects general screen share like browsing through the computer, webpages, etc. during a screen share session.
Speaker Face
Detect when the speakers face is displayed in the meeting.

modelTypeConfiguration

Keys
Value
modelType
screen_activity
modelConfig
Model Configuration object for screen_activity

Example Request

Curl
Python
1
curl --location --request POST 'https://api.marsview.ai/cb/v1/conversation/compute' \
2
--header 'Content-Type: application/json' \
3
--header "authorization: {{Insert Auth Token}}" \
4
--data-raw '{
5
"txnId": "{{Insert txn ID}}",
6
"enableModels":[
7
"modelType":"screengrabs",
8
"modelConfig":{
9
"ocr":{
10
"enable":true
11
},
12
"screen_activity" : {
13
"enable" : true,
14
"classes" : ["drawing","ppt","screen',"speaker_face","speaker_presentation"]
15
}
16
}
17
},
18
{
19
"modelType":"screen_activity"
20
}
21
]
22
}'
Copied!
1
import requests
2
auth_token = "replace this with your auth token"
3
txn_id = "Replace this with your txn id"
4
request_url = "https://api.marsview.ai/cb/v1/conversation/compute"
5
6
7
def get_screen_activity():
8
payload={
9
"txnId": txn_id,
10
"enableModels":[
11
{
12
"modelType":"screengrabs",
13
"modelConfig":{
14
"ocr":{
15
"enable":True
16
}
17
}
18
},
19
{
20
"modelType":"screen_activity"
21
}
22
]
23
}
24
headers = {'authorization': '{}'.format(auth_token)}
25
26
response = requests.request("POST",headers=headers, json=payload)
27
print(response.text)
28
if response.status_code == 200 and response.json()["status"] == "true":
29
return response.json()["data"]["enableModels"]["state"]["status"]
30
else:
31
raise Exception("Custom exception")
32
33
if __name__ == "__main__":
34
get_screen_activity()
Copied!

Example Response

1
"data": {
2
"meetingActivity": [
3
{
4
"shotId": 1,
5
"startFrame": 1,
6
"endFrame": 3237,
7
"meetingActivity": [
8
{
9
"className": "speaker_fused"
10
}
11
],
12
"meetingActivityAccuracy": {
13
"speakerFace": "3.0225227"
14
},
15
"startTime": 66.66666666666667,
16
"endTime": 215800
17
}
18
}
Copied!

Response Object

Field
Description
meetingActivity
List of meeting activity objects
shotId
Id of the collection of frames being analyzed
startFrame
First frame of the specified shotId
endFrame
Last frame of the specified shotId
meetingActivity[meetingActivity]
Type of meeting activity identified in the shotId
className
Type of class best describing the type of screen activity
meetingActivityAccuracy
Object speicying the accuracy of the predicted meeting activity
speakerFace
startTime
Start time of the shotId in milliseconds
endTime
Start time of the shotId in milliseconds
Last modified 1mo ago