Retrieve Ingested Data for a Specific Segment via REST API
In some cases, users might wish to investigate their data and get the contextual ingested data of a specific data segment, in order to check a hypostasis, an insight they received, or to explore their data for any other reason.
For this, users can now leverage Mona's REST API to retrieve said data in JSON format, showing all context instances in this segment. This can now be easily achieved with Mona's /get_ingested_data REST API endpoint.
To access Mona's API, first, an accessToken must be generated and added to the request.
Once users have their accessToken they can add it to their authorization and send a POST request with the following params:
"contextClass" - The context class of the desired segment
"startTime" - the start time in epoch timestamp
"endTime" - the end time jn epoch timestamp
"segment" - the specific segment - required format is SegmentType
"samplingThreshold" - the amount of records to return. As the records are sampled from the data, the amount is approximate.
Optional params:
"excludedSegments" - specific segments in the data you wish to exclude
curl --location --request POST 'https://api{YOUR-USER-ID}.monalabs.io/get_ingested_data' \
--header 'Authorization: Bearer YOUR-ACCESS-TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"contextClass": "LOAN_APPLICATION_TUTORIAL",
"startTime": "1611757624",
"endTime": "1619619754",
"segment": {
"occupation": [
{
"value": "retail"
}
]
},
"excludedSegments": [
{
"city": [
{
"value": "SpecialValueSemantics.MISSING"
}
]
}
],
"samplingThreshold": 10000
}'
import requests
url = "https://api{YOUR-USER-ID}.monalabs.io/get_ingested_data"
payload={
"contextClass": "LOAN_APPLICATION_TUTORIAL",
"startTime": "1611757624",
"endTime": "1619619754",
"segment": {
"occupation": [
{
"value": "retail"
}
]
},
"excludedSegments": [
{
"city": [
{
"value": "SpecialValueSemantics.MISSING"
}
]
}
],
"samplingThreshold": 10000
}
headers = {
'Authorization': 'Bearer YOUR-ACCESS-TOKEN',
'Content-Type': 'application/json'
}
response = requests.request("POST", url, headers=headers, json=payload)
print(response.text)
from mona_sdk.client import Client
api_key = <YOUR-API-KEY>
secret = <YOUR-SECRET>
my_mona_client = Client(api_key, secret)
my_mona_client.get_ingested_data_for_a_specific_segment(
context_class="LOAN_APPLICATION_TUTORIAL",
start_time=1611757624,
end_time=1619619754,
segment={"occupation": [{"value": "retail"}]},
excluded_segments=[{"city": [{"value": "SpecialValueSemantics.MISSING"}]}],
sampling_threshold=10000
)
The response will be a list of all the fields which exist in at least one instance in the response, and an array of contexts that match the defined params, along with their field values.
Max segment size
The maximum size of the response will be 50,000. For segments larger the 50k, the response will be a 50k size sample of the segment. In order to get the entire segment, try to split the time range into sections.
{
"response_data": {
"keys": [
"context",
"context_class",
"context_class_contexts.LOAN_APPLICATION_TUTORIAL",
"fields.TIME",
"fields.approved_amount_numeric",
"fields.city_in_state_string",
"fields.city_string",
"fields.feature_0_numeric",
"fields.feature_1_numeric",
"fields.feature_2_numeric",
"fields.feature_3_numeric",
"fields.feature_4_numeric",
"fields.feature_5_numeric",
"fields.feature_6_numeric",
"fields.feature_7_numeric",
"fields.feature_8_numeric",
"fields.feature_9_numeric",
"fields.loan_paid_back_boolean",
"fields.loan_taken_boolean",
"fields.model_version_string",
"fields.occupation_string",
"fields.offered_amount_numeric",
"fields.offered_approved_delta_normalized_numeric",
"fields.offered_approved_delta_numeric",
"fields.purpose_string",
"fields.return_until_numeric",
"fields.stage_string",
"fields.state_string",
"last_updated",
"normalized_hash",
"timestamp",
"user_id",
"version_timestamp"
],
"crcs": [
{
"context": "360438a45f951e81367360125b0afabf",
"user_id": "84fbc309-0773-4a65-ab08-61fac32de3e3",
"context_class": "LOAN_APPLICATION_TUTORIAL",
"timestamp": 1617444045000,
"last_updated": 1635759100727,
"fields.occupation_string": "retail",
"fields.city_string": "Kentucky_Louisville",
"fields.state_string": "Kentucky",
"fields.purpose_string": "Surgery",
"fields.loan_taken_boolean": true,
"fields.return_until_numeric": 1618653645000.0,
"fields.offered_amount_numeric": 3538.0,
"fields.approved_amount_numeric": 3538.0,
"fields.feature_0_numeric": 2141694.7963954243,
"fields.feature_1_numeric": 0.6120260477934308,
"fields.feature_2_numeric": 15.0,
"fields.feature_3_numeric": 6452.0,
"fields.feature_4_numeric": 0.24795049940804026,
"fields.feature_5_numeric": 0.9120360148259078,
"fields.feature_6_numeric": 2.0,
"fields.feature_7_numeric": 1.4686977356388236,
"fields.feature_8_numeric": 717.0736151662062,
"fields.feature_9_numeric": 5.963846902166335,
"fields.stage_string": "inference",
"fields.model_version_string": "v1",
"fields.loan_paid_back_boolean": true,
"fields.offered_approved_delta_numeric": 0.0,
"fields.offered_approved_delta_normalized_numeric": 0.0,
"fields.city_in_state_string": "Kentucky_Louisville_Kentucky",
"fields.TIME": 1617444045000,
"version_timestamp": 1633013311921,
"context_class_contexts.LOAN_APPLICATION_TUTORIAL": "360438a45f951e81367360125b0afabf",
"normalized_hash": 0.3521298
},
{
"context": "ca626acd8706dc6c515830becd6749ed",
"user_id": "84fbc309-0773-4a65-ab08-61fac32de3e3",
"context_class": "LOAN_APPLICATION_TUTORIAL",
"timestamp": 1618132219000,
"last_updated": 1635759100727,
"fields.occupation_string": "retail",
"fields.city_string": "Indiana_Fort Wayne",
"fields.state_string": "Indiana",
"fields.purpose_string": "Investment",
"fields.loan_taken_boolean": true,
"fields.return_until_numeric": 1619341819000.0,
"fields.offered_amount_numeric": 1937.0,
"fields.approved_amount_numeric": 1937.0,
"fields.feature_0_numeric": 1701923.759697594,
"fields.feature_1_numeric": 1.094079226981069,
"fields.feature_2_numeric": 509.0,
"fields.feature_3_numeric": 6351.0,
"fields.feature_4_numeric": 0.17935152796028775,
"fields.feature_5_numeric": 0.5554486132131119,
"fields.feature_6_numeric": 20.0,
"fields.feature_7_numeric": 1.466018986391044,
"fields.feature_8_numeric": 814.8875196389081,
"fields.feature_9_numeric": 35.91066808595547,
"fields.stage_string": "inference",
"fields.model_version_string": "v1",
"fields.loan_paid_back_boolean": true,
"fields.offered_approved_delta_numeric": 0.0,
"fields.offered_approved_delta_normalized_numeric": 0.0,
"fields.city_in_state_string": "Indiana_Fort Wayne_Indiana",
"fields.TIME": 1618132219000,
"version_timestamp": 1633013313820,
"context_class_contexts.LOAN_APPLICATION_TUTORIAL": "ca626acd8706dc6c515830becd6749ed",
"normalized_hash": 0.3520168
},
{
"context": "c6d42c876daca15fba4df6f838c883f3",
"user_id": "84fbc309-0773-4a65-ab08-61fac32de3e3",
"context_class": "LOAN_APPLICATION_TUTORIAL",
"timestamp": 1612102976000,
"last_updated": 1635759100727,
"fields.occupation_string": "retail",
"fields.city_string": "Nevada_Las Vegas",
"fields.state_string": "Nevada",
"fields.purpose_string": "Home construction",
"fields.loan_taken_boolean": false,
"fields.offered_amount_numeric": 2060.0,
"fields.feature_0_numeric": 2733883.902599836,
"fields.feature_1_numeric": 2.590430532072283,
"fields.feature_2_numeric": 148.0,
"fields.feature_3_numeric": 6009.0,
"fields.feature_4_numeric": 0.12605219573000648,
"fields.feature_5_numeric": 1.0707352975549105,
"fields.feature_6_numeric": 73.0,
"fields.feature_7_numeric": 22.602342489686187,
"fields.feature_8_numeric": 699.6697312559934,
"fields.feature_9_numeric": 9.040605384079326,
"fields.stage_string": "inference",
"fields.model_version_string": "v1",
"fields.city_in_state_string": "Nevada_Las Vegas_Nevada",
"fields.TIME": 1612102976000,
"version_timestamp": 1633013360391,
"context_class_contexts.LOAN_APPLICATION_TUTORIAL": "c6d42c876daca15fba4df6f838c883f3",
"normalized_hash": 0.351934
}
]
}
}
Updated over 1 year ago