Field Build Functions
These functions can be used in order to build and manipulate the fields
abs_value
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
abs_value | Returns the absolute value for a given numeric value. | numeric_value (type: Number) | Number |
{
"abs_value_of_source": {
"type": "numeric",
"function": "abs_value",
"source": "source_name"
}
}
{
"source_name": -6
}
{
"abs_value_of_source": 6
}
all_checks
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
all_checks | A directive to check for multiple conditions. Receives a sequence of values, returns True iff all the values return True. Note that it is possible and advised to use this method over results of other boolean field functions (such as check_existence, equal_const and negative_check) in order to create complex boolean fields. Not suitable for a boolean array source, only for multiple boolean sources. If remove_if_null is set to True, if one of the boolean_values is null, this field will be removed altogether. | boolean_values (type: Sequence[Optional[bool]]) | Optional[bool] | remove_if_null |
{
"all_sources_are_true": {
"type": "boolean",
"function": "all_checks",
"sources": [
"source_name_1",
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": true,
"source_name_2": true
}
<EXAMPLE OUTPUT>
{
"all_sources_are_true": true
}
all_values_field_in_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
all_values_field_in_array | A directive which returns an array of values from all the occurrences of a key in an array of dictionaries. | array_value (type: Sequence[Mapping]) | Sequence | key (type: str) |
{
"values_from_objects_in_array": {
"type": "numeric",
"function": "all_values_field_in_array",
"source": "source_name",
"args": [
"name"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "john",
"age": 13
},
{
"name": "bob",
"age": 20
}
]
}
<EXAMPLE OUTPUT>
{
"values_from_objects_in_array": [
"john",
"bob"
]
}
area
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
area | A directive for calculating the area of polygon. The input should be a polygon in the format of a GeoJson (json or WKT) or a TopoJson. The returned area will be in square meter (m^2). Set convert_to_km to true in order to get the result in square kilometer(km^2) | polygon | convert_to_km |
{
"area_of_polygon": {
"type": "numeric",
"function": "area",
"source": "polygon"
}
}
<EXAMPLE INPUT 1>
{
"polygon": {
"type": "Polygon",
"coordinates": [[[-50,-90],[-50,90],[50,90],[-50,-90]]]
}
}
<EXAMPLE INPUT 2>
{
"polygon": "POLYGON ((-50 -90, -50 90, 50 90, 50 -90, -50 -90))"
}
<EXAMPLE OUTPUT >
{
"area_of_polygon": 18000.0
}
array_intersection
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
array_intersection | A directive for getting the intersection of two arrays | first_array_value (type: Sequence) second_array_value (type: Sequence) | Sequence |
{
"array_from_intersected_arrays": {
"type": "string",
"function": "array_intersection",
"sources": [
"source_name_1",
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": [7,1,5,2,3,6],
"source_name_2": [3,8,6,20,7]
}
<EXAMPLE OUTPUT>
{
"array_from_intersected_arrays": [3,6,7]
}
array_length
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
array_length | A directive for creating a field according to the length of an array value. | array_value (type: Sequence) | int | default_value |
{
"length_of_array": {
"type": "numeric",
"function": "array_length",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [1,2,3,4,6,9,22]
}
<EXAMPLE OUTPUT>
{
"length_of_array": 7
}
array_numbers
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
array_numbers | A directive for creating an array which consists only of the numeric values of a given array. | array_value (type: Sequence) | Sequence[Number] | default_value |
{
"numeric_values": {
"type": "numeric",
"function": "array_numbers",
"source": "mixed_array"
}
}
<EXAMPLE INPUT>
{
"mixed_array": [1,5,3,true,"dog"]
}
<EXAMPLE OUTPUT>
{
"numeric_values": [1,5,3]
}
array_top_value
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
array_top_value | A directive for creating a field according to the max numeric value of an array. Note: given array must contain only numeric values. | array_value (type: Sequence[Number]) | Optional[Number] | default_value |
{
"top_value_of_array": {
"type": "numeric",
"function": "array_top_value",
"source": "array_source"
}
}
<EXAMPLE INPUT>
{
"array_source": [1,5,32,3,2.8]
}
<EXAMPLE OUTPUT>
{
"top_value_of_array": 32
}
array_tops_delta
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
array_tops_delta | A directive for creating a field according to the delta between the two largest numeric values of an array. Note: given array must contain only numeric values. | array_value (type: Sequence[Number]) | Optional[Number] | default_value |
{
"delta_of_array_top_values": {
"type": "numeric",
"function": "array_top_value",
"source": "array_source"
}
}
<EXAMPLE INPUT>
{
"array_source": [32,1,5,12]
}
<EXAMPLE OUTPUT>
{
"delta_of_array_top_values": 20
}
arrays_product
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
arrays_product | A directive which takes arrays and returns the cartesian product of those arrays, with its values concatenated to one string for each subset. | arrays_value (type: Sequence) | Sequence[str] | separator (type: str) avoid_same_values (type: bool) |
{
"arrays_product": {
"type": "string",
"sources": ["source1", "source2"]
"function": "arrays_product",
"args": [
"_"
]
}
}
<EXAMPLE INPUT>
{
"source1": [1,2,3]
"source2": ["a", "b"]
}
<EXAMPLE OUTPUT>
{
"arrays_product": ['1_b', '1_a', '2_b', '2_a', '3_b', '3_a']
}
averages_delta
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
averages_delta | Returns the delta between the average value of the two arrays. | first_array_value (type: Sequence[Optional[Number]]) second_array_value (type: Sequence[Optional[Number]]) | Optional[Number] |
{
"delta_of_averages_of_two_arrays": {
"type": "numeric",
"function": "averages_delta",
"sources": [
"source_name_1",
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": [3,7,10,20],
"source_name_2": [1,5,9]
}
<EXAMPLE OUTPUT>
{
"delta_of_averages_of_two_arrays": 5
}
boolean_via_numeric
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
boolean_via_numeric | Return the boolean value of the numeric value that the given str describes. | value (type: str) | bool |
{
"boolean_value_field": {
"type": "boolean",
"function": "boolean_via_numeric",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": "0"
}
<EXAMPLE OUTPUT>
{
boolean_value_field": false
}
clean_null_str
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
clean_null_str | A workaround directive to nullify "null" strings coming by mistake from the user. | actual_value |
{
"nullified_field_name": {
"type": "string",
"source": "source_name",
"function": "clean_null_str"
},
"not_nullified_field_name": {
"type": "string",
"source": "source_name_2",
"function": "clean_null_str"
}
<EXAMPLE INPUT>
{
"source_name": "null",
"source_name_2": "dog"
}
<EXAMPLE OUTPUT>
{
"nullified_field_name": null,
"not_nullified_field_name": "dog"
}
concat
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
concat | A directive which takes a sequence field, and returns its values concatenated as a string. | actual_value (type: Union[Mapping, Sequence]) | Optional[str] |
{
"concatenated_array": {
"type": "string",
"function": "concat",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [
"John",
"Bob",
"Sharon",
"Joe"
]
}
<EXAMPLE OUTPUT>
{
"concatenated_array": [
"John_Bob_Sharon_Joe"
]
}
concat_inner_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
concat_inner_array | Returns an array concatenation of all values in the the inner_array_key in the objects originally found in the source array. | outer_array_value (type: Sequence[Mapping]) | Sequence | inner_array_key (type: str) |
{
"array_of_concatenated_arrays": {
"type": "string",
"function": "concat_inner_array",
"source": "source_name",
"args": [
"a"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"a": [1,2,3],
"b": ["a","b","c"]
},
{
"a": [4,5,6],
"b": ["d","e","f"]
}
]
}
<EXAMPLE OUTPUT>
{
"array_of_concatenated_arrays": ["1","2","3","4","5","6"]
}
concat_strings
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
concat_strings | A directive for concatenating all the given string/strings sequences sources. If sources are strings sequences, concat each index of the sources. If values contains str arrays, in case of a None arg in one of the arrays, the returned value in the corresponding index will be None. | values (type: Sequence[Union[str, Sequence[str]]]) | Union[str, Sequence[str]] | separator (type: str) |
{
"concatenated_string": {
"type": "string",
"function": "concat_strings",
"sources": [
"source_name_1",
"source_name_2"
],
"args": [
", "
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": "Jack",
"source_name_2": "Black"
}
<EXAMPLE OUTPUT>
{
"concatenated_string": "Jack, Black"
}
concat_strings_recursive_arrays
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
concat_strings_recursive_arrays | A directive for generating a sequence of strings, concatanated from keys of objects within arrays. It currently has a very specific usage: It expects a source which is an array of objects, then a sequence of keys, the first of which should be a key to look for a string value, the second should be a key to look for a new array of objects, the third is again a key for a string value within this new array, and the forth is again a key for looking for a new array of objects, and so on. | top_array_value (type: Sequence[Optional[Mapping]]) | Optional[str] | internal_keys (type: Sequence[str]) separator (type: str) |
{
"concatenated_strings_in_recursive_arrays": {
"type": "string",
"source": "source_name",
"function": "concat_strings_recursive_arrays",
"args": [
[
"feature_id",
"attributes",
"attribute_1",
"values",
"unique_id"
]
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"feature_id": 3833,
"attributes": [
{
"attribute_1": 1242,
"values": [
{
"unique_id": 12
}
]
}
]
},
{
"feature_id": 2336,
"attributes": [
{
"attribute_1": 923,
"values": [
{
"unique_id": 132
}
]
},
{
"attribute_1": 1759,
"values": [
{
"unique_id": 243
}
]
}
]
}
]
}
<EXAMPLE OUTPUT>
{
"concatenated_strings_in_recursive_arrays": [
"3833_1242_12",
"2336_923_132",
"2336_1759_243"
]
}
const_is_in_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
const_is_in_array | Return true if the given array contains the given constant value, false otherwise. | array_value (type: Sequence) | bool | value |
{
"does_source_contains_const": {
"type": "boolean",
"function": "const_is_in_array",
"source": array_source,
"args": [
"blue"
]
}
}
<EXAMPLE INPUT>
{
"array_source": [red, yellow, ,blue, orange]
}
<EXAMPLE OUTPUT>
{
"does_source_contains_const": true
}
convert_to_float
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
convert_to_float | Convert string to float safely (if it's already int/float - no conversion needed). In case the given value is not representing a number the return value is None. Values in null_values are replaced with null. | original_value (type: Optional[Number]) | Number | null_values |
count_characters
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
count_characters | A directive which gets a string value or array, and returns the amount of characters (if array - for each element). | string_or_array_value | int |
{
"count_characters_field_name": {
"type": "int",
"function": "count_characters",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": ["Hi there, my name is John.", "Hello"]
}
<EXAMPLE OUTPUT>
{
"count_characters_field_name": [26, 5]
}
count_occurrences_in_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
count_occurrences_in_array | A directive that, given an array field, counts the amount of items in that array in which the value for given key is equal to the given inner_value. | array_value (type: Sequence[Mapping]) | int | key (type: str) inner_value default_value |
{
"number_of_value_occurrences": {
"type": "numeric",
"function": "count_occurrences_in_array",
"source": "source_name",
"args": [
"name",
"Bob"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 13
},
{
"name": "Bob",
"age": 20
},
{
"name": "Sharon",
"age": 17
},
{
"name": "Bob",
"age": 10
}
]
}
<EXAMPLE OUTPUT>
{
"number_of_value_occurrences": 2
}
count_recursively
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
count_recursively | A directive which counts all internal elements in the given array, recursively. | array_value (type: Sequence) | int |
{
"number_of_elements_recursively": {
"type": "numeric",
"function": "count_recursively",
"source": "source_name"
}
}
{
"source_name": [
"bob",
[
"dog",
"cat"
],
"sharon"
]
}
{
"number_of_elements_recursively": 4
}
count_words
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
count_words | A directive which gets a string value or array, and returns the amount of words (if array - for each element). The amount of words is calculated based on the standard python split method without args, i.e. words are separated by any number and form of whitespace chars. | string_or_array_value | int |
{
"count_words_field_name": {
"type": "int",
"function": "count_words",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": ["Hi there, my name is John.", "Hello"]
}
<EXAMPLE OUTPUT>
{
"count_words_field_name": [6, 1]
}
day_of_week_from_timestamp_seconds
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
day_of_week_from_timestamp_seconds | Returns the day of week (UTC) from the given timestamp. | timestamp_value (type: Number) |
{
"day_of_timestamp": {
"type": "string",
"function": "day_of_week_from_timestamp_seconds",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": 1610569800
}
<EXAMPLE OUTPUT>
{
"day_of_timestamp": "Wednesday"
}
dedup_array_of_objects_by_key
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
dedup_array_of_objects_by_key | Returns the given array, deduped by the values in the given key. | input_value (type: Union[Sequence[dict], dict]) | Sequence | dedup_key (type: str) |
{
"deduped_sub_array": {
"type": "string",
"function": "dedup_array_of_objects_by_key",
"source": "source_name",
"args": [
"name"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 20
},
{
"name": "Bob",
"age": 30
},
{
"name": "Max",
"age": 19
},
{
"name": "Bob",
"age": 10
}
]
}
<EXAMPLE OUTPUT>
{
"deduped_sub_array": [
{
"name": "John",
"age": 20
},
{
"name": "Bob",
"age": 30
},
{
"name": "Max",
"age": 19
}
]
}
delta
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
delta | A directive for calculating delta of two values. | first_value (type: Optional[Number]) second_value (type: Optional[Number]) | Optional[Number] | default_value |
{
"delta_between_values": {
"type": "numeric",
"sources": [
"numeric_source_1",
"numeric_source_2"
],
"function": "delta"
}
}
<EXAMPLE INPUT>
{
"numeric_source_1": 50,
"numeric_source_2": 2
}
<EXAMPLE OUTPUT>
{
"delta_between_values": 48
}
delta_between_first_and_second
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
delta_between_first_and_second | Returns the delta between the max value and the second max value from a list of (dynamic) multiple numeric value sources. Not suitable for numerical array sources. | numeric_values (type: Sequence[Optional[Number]]) | Optional[Number] |
{
"delta_of_two_highest_values": {
"type": "numeric",
"function": "delta_between_first_and_second",
"sources": [
"source_name_1",
"source_name_2",
"source_name_3"
]
}
}
{
"source_name_1": 5,
"source_name_2": 10,
"source_name_3": 7
}
{
"delta_of_two_highest_values": 3
}
delta_from_const
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
delta_from_const | A directive for calculating delta between a given number and a const. | first_value (type: Optional[Number]) | Optional[Number] | const_to_subtract (type: Number) default_value |
{
"delta_between_source_and_const": {
"type": "numeric",
"function": "delta_from_const",
"source": "numeric_source_name",
"args": [
10
]
}
}
<EXAMPLE INPUT>
{
"numeric_source_name": 50
}
<EXAMPLE OUTPUT>
{
"delta_between_source_and_const": 40
}
distance
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
distance | A directive for calculating the distance between two points. The input should be two given Geopoints. The returned distance will be in meters (m). Set convert_to_km to true in order to get the result in kilometers (km) | point1 point2 | convert_to_km |
{
"distance_between_two_points": {
"function": "distance",
"sources": ["point1", "point2"],
"type": "numeric"
}
}
<EXAMPLE INPUT>
{
[{
"point1": {"location" : "POINT (-71.34 41.12)" },
"point2": {"location" : "POINT (52.406374 16.9251681)" }
}]
}
<EXAMPLE OUTPUT>
{
"distance_between_two_points": 11365307.89
}
divide
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
divide | A directive for dividing one value by another. | numerator_value (type: Optional[Number]) denominator_value (type: Optional[Number]) | Optional[Number] |
{
"quotient_of_division": {
"type": "numeric",
"function": "divide",
"sources": [
"numeric_source_name_1",
"numeric_source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"numeric_source_name_1": 50,
"numeric_source_name_2": 10
}
<EXAMPLE OUTPUT>
{
"quotient_of_division": 5
}
divide_by_const
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
divide_by_const | A directive for dividing one value by a const. | numerator_value (type: Optional[Number]) | Optional[Number] | denominator (type: Optional[Number]) |
{
"quotient_of_division_with_const": {
"type": "numeric",
"function": "divide_by_const",
"source": "numeric_source_name",
"args": [
"5"
]
}
}
<EXAMPLE INPUT>
{
"numeric_source_name_": 50
}
<EXAMPLE OUTPUT>
{
"quotient_of_division_with_const": 10
}
dynamic_field_value_from_object
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
dynamic_field_value_from_object | Returns the value of a dynamically chosen field within an object. | dict_object_value (type: Mapping) field_name_value |
{
"value_of_dynamic_field": {
"type": "string",
"function": "dynamic_field_value_from_object",
"sources": [
"source_name",
"field_key_name"
]
}
}
<EXAMPLE INPUT>
{
"source_name": {
"name": "Bob",
"age": 23,
"city": "New York"
},
"field_key_name": "city"
}
<EXAMPLE OUTPUT>
{
"value_of_dynamic_field": "New York"
}
empty_to_null
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
empty_to_null | A directive which is just like the identity function for strings, only it returns "null" for empty strings. | original_value (type: str) | str |
{
"example_field_name": {
"type": "string",
"function": "empty_to_null",
"sources": [
"source_name"
]
},
"empty_field_name": {
"type": "string",
"function": "empty_to_null",
"sources": [
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name": "dog",
"source_name_2": ""
}
<EXAMPLE OUTPUT>
{
"example_field_name": "dog",
"empty_field_name": null
}
equality_check
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
equality_check | A directive to check for equality between two values. | first_value second_value | Optional[bool] |
{
"is_equale": {
"type": "boolean",
"function": "equality_check",
"sources": [
"source_name_1",
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": 10,
"source_name_2": 10
}
<EXAMPLE OUTPUT>
{
"is_equale": true
}
equality_check_const
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
equality_check_const | A directive to check for equality between a value and a constant value. | value | const |
{
"is_equal_to_const": {
"type": "boolean",
"function": "equality_check_const",
"source": "source_name",
"args": [
10
]
}
}
<EXAMPLE INPUT>
{
"source_name": 10
}
<EXAMPLE OUTPUT>
{
"is_equal_to_const": true
}
existence_check
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
existence_check | A directive to check for an existence of a value. | value | bool | non_existent_value |
{
"value_exists": {
"type": "boolean",
"function": "existence_check",
"source": "source_name"
},
"value_exists_2": {
"type": "boolean",
"function": "existence_check",
"source": "source_name_2"
},
"value_exists_3": {
"type": "boolean",
"function": "existence_check",
"source": "source_name_3"
}
}
<EXAMPLE INPUT>
{
"source_name": "John",
"source_name_2": null
}
<EXAMPLE OUTPUT>
{
"value_exists": true,
"value_exists_2": false,
"value_exists_3": false
}
field_delta
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
field_delta | A directive to get a field delta between the first and second values of a specific key in an array field. NOTE: If no second value is found, we use zero instead. | array_value (type: Sequence[Mapping[str, Number]]) | Optional[Number] | key (type: str) sort (type: bool) reversed_sort (type: bool) default_value |
{
"delta_of_first_and_second_values": {
"type": "string",
"function": "field_delta",
"source": "source_name",
"args": [
"age"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 20
},
{
"name": "Bob",
"age": 13
},
{
"name": "Sharon",
"age": 17
}
]
}
<EXAMPLE OUTPUT>
{
"delta_of_first_and_second_values": 7
}
field_value_from_object
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
field_value_from_object | Returns the value of a field within an object | dict_object_value (type: Mapping) | field_name |
{
"city_name_from_obj": {
"type": "string",
"function": "field_value_from_object",
"source": "source_name",
"args": [
"city"
]
}
}
<EXAMPLE INPUT>
{
"source_name": {
"name": "Bob",
"age": 23,
"city": "New York"
}
}
<EXAMPLE OUTPUT>
{
"city_name_from_obj": "New York"
}
filter_according_to_boolean_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
filter_according_to_boolean_array | Returns the original array, filtered according to a same-length array of boolean values. | original_array_value boolean_array_value | should_invert |
{
"filtered_array": {
"type": "string",
"function": "filter_according_to_boolean_array",
"sources": [
"array_source_name",
"boolean_array_source_name"
]
}
}
<EXAMPLE INPUT>
{
"array_source_name": [
"cat",
"cow",
"dog",
"sheep"
],
"boolean_array_source_name": [
true,
true,
false,
true
]
}
<EXAMPLE OUTPUT>
{
"filtered_array": [
"cat",
"cow",
"sheep"
]
}
filter_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
filter_array | Returns a sub array for the given array of objects, according to whether the value in the given key of the object is from the possible values parameter. If the bool_evaluation parameter is set, will cast the value of the given key to bool before checking. if the key parameter is None, will evaluate the given values in the array directly, and will assume they are not dictionaries. The returned value is still an array of objects, and should therefore never be "saved" in the schema (use "do_not_save": true). | array_value (type: Sequence[Mapping]) | Sequence | key (type: str) possible_values (type: Sequence) bool_evaluation (type: bool) |
{
"filtered_sub_array": {
"type": "string",
"function": "filter_array",
"source": "source_name",
"args": [
"name",
[
"John",
"Bob"
]
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 13
},
{
"name": "Bob",
"age": 20
},
{
"name": "Sharon",
"age": 17
}
]
}
<EXAMPLE OUTPUT>
{
"filtered_sub_array": [
{
"name": "John",
"age": 13
},
{
"name": "Bob",
"age": 20
}
]
}
filter_array_by_blacklist
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
filter_array_by_blacklist | Returns a sub array for the given array of objects, according to whether the value in the given key of the object is not from the blacklisted_values. The returned value is still an array of objects, and should therefore never be "saved" in the schema (use "do_not_save": true). | array_value (type: Sequence[Mapping]) | Sequence[Mapping] | key (type: str) blacklisted_values (type: Sequence) |
{
"filtered_sub_array": {
"type": "string",
"function": "filter_array_by_blacklist",
"source": "source_name",
"args": [
"name",
[
"Bob",
"Sharon"
]
],
"do_not_use": true
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 14
},
{
"name": "Bob",
"age": 33
},
{
"name": "Max",
"age": 45
}
]
}
<EXAMPLE OUTPUT>
{
"filtered_sub_array": [
{
"name": "John",
"age": 14
},
{
"name": "Max",
"age": 45
}
]
}
filter_array_by_valid_ranges
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
filter_array_by_valid_ranges | Returns a sub array for the given array of objects, according to whether the value in the given key of the object is in at least one of the given valid_ranges. If the given key is None, then this directive assumes the array isn't of objects, but of the actual numbers needed to be filtered according to the ranges. valid_ranges should be an array of pairs, which represent the minimum (inclusive) and maximum (exclusive) for each range. null (pythonic: None) can be used to suggest no minimum or no maximum. | array_value (type: Sequence[Union[Mapping[str, Number], Number]]) | Sequence | key (type: str) valid_ranges (type: Sequence[Sequence[Number]]) |
{
"filtered_sub_array": {
"function": "filter_array_by_valid_ranges",
"source": "source_name",
"args": [
"age",
[
[12,18],
[30,35]
]
],
"do_not_save": true
},
"filtred_sub_array_length": {
"type": "numeric",
"function": "array_length",
"source": "filtered_sub_array"
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 13
},
{
"name": "Bob",
"age": 20
},
{
"name": "Sharon",
"age": 17
},
{
"name": "Bob",
"age": 10
}
]
}
<EXAMPLE OUTPUT>
{
"filtered_sub_array": [
{
"name": "John",
"age": 13
},
{
"name": "Sharon",
"age": 17
}
],
"filtred_sub_array_length": 2
}
filter_array_static_key_and_dynamic_value
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
filter_array_static_key_and_dynamic_value | Returns a sub array for the given array of objects, according to whether the value in the given key of the object is equal to the compared value given dynamically as a source. The returned value is still an array of objects, and should therefore never be "saved" in the schema (use "do_not_save": true). | array_value (type: Sequence[Mapping]) compared_value | Sequence | key (type: str) |
{
"dynamically_filtered_sub_array": {
"type": "string",
"function": "filter_array_static_key_and_dynamic_value",
"sources": [
"array_value_source_name",
"dynamic_value_source_name"
]
"args": [
"name"
]
}
}
<EXAMPLE INPUT>
{
"array_value_source_name": [
{
"name": "John",
"age": 13
},
{
"name": "Bob",
"age": 20
},
{
"name": "Sharon",
"age": 17
}
],
"dynamic_value_source_name": "John"
}
<EXAMPLE OUTPUT>
{
"dynamically_filtered_sub_array": [
{
"name": "John",
"age": 13
}
]
}
filter_duplicates
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
filter_duplicates | A directive for getting a new array, with filtered duplicates. If the given value is not an array - return the value as is. | array_value (type: Sequence) | Sequence |
{
"filtered_sub_array": {
"type": "string",
"function": "filter_duplicates",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [1,2,2,2,3,4]
}
<EXAMPLE OUTPUT>
{
"filtered_sub_array": [1,2,3,4]
}
first_in_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
first_in_array | Returns the first value in an array. | array_value (type: Sequence) | default_value |
{
"first_value_in_array": {
"type": "string",
"function": "first_in_array",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [
"John",
"Bob",
"Max"
]
}
<EXAMPLE OUTPUT>
{
"first_value_in_array": "John"
}
flatten_array_recursive
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
flatten_array_recursive | A directive to flatten an array with nested arrays to one level. | array_value (type: Sequence) | Sequence |
{
"flattened_array": {
"type": "string",
"function": "flatten_array_recursive",
"source": "array_source_name"
}
}
{
"array_source_name": [
"bob",
[
"dog",
"cat"
],
"sharon"
]
}
{
"flattened_array": [
"bob",
"dog",
"cat",
"sharon"
]
}
format_string
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
format_string | A directive for string formatting. If some of the input values are arrays, this function will output an array for all the possible combinations of the inputs. | input_values (type: Sequence) | str | template (type: str) |
{
"formatted_string_field": {
"type": "string",
"function": "format_string",
"sources": [
"source_name_1",
"source_name_2"
],
"args": [
"{} and {} are co-workers"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": "John",
"source_name_2": "Bob
}
<EXAMPLE OUTPUT>
{
"formatted_string_field": "John and Bob are co-workers"
}
from_array_index
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
from_array_index | Returns the value in the given index in the given array. | array_value | index |
{
"value_of_index": {
"type": "numeric",
"function": "from_array_index",
"source": array_source,
"args": [
2
]
}
}
<EXAMPLE INPUT>
{
"array_source": [10,12,6,3,11]
}
<EXAMPLE OUTPUT>
{
"value_of_index": 6
}
geo_intersect
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
geo_intersect | A directive for calculating the intersection of two given polygons. The input should be GeoJson or TopoJson format or WKT (as a json or a json string) and the output will be returned as WKT string. | polygon1 polygon2 |
{
"intersection_of_polygons": {
"function": "geo_intersect",
"sources": ["poly1", "poly2"],
"type": "string"
}
}
<EXAMPLE INPUT>
{
[{
"poly1": {
"type": "Polygon",
"coordinates": [[
[-180,-90],
[-180,90],
[180,90],
[180,-90],
[-180,-90]
]]
},
"poly2": {
"type": "Polygon",
"coordinates": [[
[-122.801742, 45.48565],
[-122.801742, 45.60491],
[-122.584762, 45.60491],
[-122.584762, 45.48565],
[-122.801742, 45.48565]
]]
}
}]
}
<EXAMPLE OUTPUT>
{
""intersection_of_polygons": POLYGON ((-122.801742 45.60491,
-122.584762 45.60491,
-122.584762 45.48565,
-122.801742 45.48565,
-122.801742 45.60491
))"
}
get_dict_key_on_check
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
get_dict_key_on_check | Return the value of a dict's key according to a check of check_key being equal to check_value (static value supplied from the configuration as an argument) | dict_objects_array_value (type: Sequence[Mapping]) | key (type: str) check_key (type: str) check_value compare_as_floats (type: bool) |
{
"value_of_key_by_args": {
"type": "string",
"source": "source_name",
"function": "get_dict_key_on_check",
"args": [
"b",
"a",
3
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
},
{
"a": 5,
"b": 6
}
]
}
<EXAMPLE OUTPUT>
{
"value_of_key_by_args": "4"
}
get_dict_key_on_dynamic_check
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
get_dict_key_on_dynamic_check | Return the value of a dict's key according to a check of check_key being equal to dynamic value supplied from a source. | check_value dict_objects_array_value (type: Sequence[Mapping]) | key (type: str) check_key (type: str) compare_as_floats (type: bool) |
{
"value_of_key_by_dynamic_check": {
"type": "string",
"sources": [
"check_value_source",
"source_name"
],
"function": "get_dict_key_on_dynamic_check",
"args": [
"b",
"a"
]
}
}
<EXAMPLE INPUT>
{
"check_value_source": 3,
"source_name": [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
},
{
"a": 5,
"b": 6
}
]
}
<EXAMPLE OUTPUT>
{
"value_of_key_by_dynamic_check": "4"
}
get_nth_highest_key
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
get_nth_highest_key | Returns the key of the nth highest value in a given dict. | dict_value (type: Mapping[str, Number]) | n (type: int) |
{
"key_of_nth_highest_value": {
"type": "numeric",
"function": "get_nth_highest_key",
"source": "source_name",
"args": [
1
]
}
}
<EXAMPLE INPUT>
{
"source_name": {
"average_age_1": 34,
"average_age_2": 33,
"average_age_3": 42
}
}
<EXAMPLE OUTPUT>
{
"key_of_nth_highest_value": "average_age_2"
}
get_nth_highest_value
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
get_nth_highest_value | Returns the nth highest value of all values in a given dict. | dict_value (type: Mapping[str, Number]) | n (type: int) |
{
"nth_highest_value_of_keys": {
"type": "numeric",
"function": "get_nth_highest_value",
"source": "source_name",
"args": [
1
]
}
}
<EXAMPLE INPUT>
{
"source_name": {
"average_age_1": 34,
"average_age_2": 33,
"average_age_3": 42
}
}
<EXAMPLE OUTPUT>
{
"nth_highest_value_of_keys": 33
}
get_top_key
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
get_top_key | Returns the key with the highest number in its value, from dictionary which maps strings to numbers. | dict_value (type: Mapping[str, Number]) |
{
"key_of_highest_value": {
"type": "string",
"function": "get_top_key",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": {
"key_1": 4,
"key_2": 12,
"key_3": 7,
"key_4": 22
}
}
<EXAMPLE OUTPUT>
{
"key_of_highest_value": "key_4"
}
get_top_value
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
get_top_value | Returns the highest value of all values in a given dict. | dict_value (type: Mapping[str, Number]) |
{
"highest_value_of_keys": {
"type": "numeric",
"function": "get_top_value",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": {
"key_1": 4,
"key_2": 12,
"key_3": 7,
"key_4": 22
}
}
<EXAMPLE OUTPUT>
{
"highest_value_of_keys": 22
}
hour_of_day_from_timestamp_seconds
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
hour_of_day_from_timestamp_seconds | Returns the hour of the day (UTC) from the given timestamp. | timestamp_value (type: Number) |
{
"hour_of_timestamp": {
"type": "numeric",
"function": "hour_of_day_from_timestamp_seconds",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": 1610569800
}
<EXAMPLE OUTPUT>
{
"hour_of_timestamp": 20
}
identity
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
identity | A default identity field build function. | value |
import_from_other_context_class
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
import_from_other_context_class | Imports the value of the field from other context class. Using the given foreign id value, fetch a corresponding existing CRC and returns the values in the foreign field. The string_to_replace_dot param allows to replace the dot character in the foreign_ids to a provided string. | foreign_ids (type: Union[str, list, tuple]) | foreign_field (type: str) foreign_context_class (type: str) foreign_id_prefix string_to_replace_dot (type: str) |
{
"imported_value": {
"type": "string",
"function": "import_from_other_context_class",
"source": "some_id",
"args": [
"name_of_field_in_other_context_class",
"NAME_OF_OTHER_CONTEXT_CLASS"
]
}
}
{
"some_id": "532n-kl52-gs53-weg5"
}
{
"imported_value": "value_of_field_in_other_context_class"
}
import_from_other_context_class_as_of
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
import_from_other_context_class_as_of | Imports the value of the field from another context class, as of a time in a given time field name in the importing CRC. This logic simulates the SQL join_asof function. NOTE: This function assumes all relevant numeric variables (in the source and in the foreign "as of" fields) are in the same format (e.g., not one in seconds and the other in milliseconds) | lookup_field_value as_of_field_value | foreign_lookup_field foreign_as_of_field foreign_imported_field foreign_context_class max_valid_range |
{
"imported_value": {
"type": "numeric",
"function": "import_from_other_context_class_as_of",
"sources": [
"lookup_field_name",
"as_of_field_name"
],
"args": [
"foreign_lookup_field",
"foreign_as_of_field",
"foreign_imported_field",
"foreign_context_class",
"max_valid_range"
]
}
}
{
"city": "Amsterdam",
"as_of": 1722169215000,
"city_name",
"updated_at",
"population",
"CITIES",
3600
}
{
"imported_value": 921402
}
is_in_dynamic_set
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
is_in_dynamic_set | A directive to check if a value is in a set of values. | value set_of_values (type: Sequence) | Optional[bool] |
{
"fields": {
"value_is_in_set": {
"type": "boolean",
"function": "is_in_dynamic_set",
"sources": ["source_name", "source_name_2"],
},
},
}
<EXAMPLE INPUT>
{
"source_name": "John",
"source_name_2": ["John", "Bob", "Frank", "Paul"]
}
<EXAMPLE OUTPUT>
{
"value_is_in_set": true
}
is_in_other_array_mapper
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
is_in_other_array_mapper | A mapper directive for boolean values according to whether each value is in another array or not | first_array_value (type: Sequence) second_array_value (type: Sequence) | Sequence[bool] |
{
"boolean_values_according_to_sources": {
"type": "boolean",
"function": "is_in_other_array_mapper",
"sources": [
"source_name_1",
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": [
"John",
"Bob",
"Sharon",
"Joe"
],
"source_name_2": [
"Max",
"Bob",
"Peter",
"Joe"
]
}
<EXAMPLE OUTPUT>
{
"boolean_values_according_to_sources": [
false,
true,
false,
true
]
}
is_in_set
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
is_in_set | A directive to check if a value is in a given values set. | value | Optional[bool] | set_key (type: str) treat_missing_as_false (type: bool) |
{
"fields": {
"value_is_in_set": {
"type": "boolean",
"function": "is_in_set",
"source": "source_name",
"args": [
"set_name"
]
},
},
"data": {
"set_name": [
"John",
"Bob",
"Frank",
"Paul"
]
}
}
<EXAMPLE INPUT>
{
"source_name": "John"
}
<EXAMPLE OUTPUT>
{
"value_is_in_set": true
}
is_monotone
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
is_monotone | A directive which checks if the given array is monotone. | array_value (type: Sequence) | bool | ascending use_weak |
{
"array_is_monotone": {
"type": "boolean",
"function": "is_monotone",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [1,3,4,6,12]
}
<EXAMPLE OUTPUT>
{
"array_is_monotone": true
}
keep_only_digits
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
keep_only_digits | A directive which takes a string (or array of strings) and returns that string only with its digits, stripping away all non-digit characters. | actual_value (type: Union[str, Sequence[str]]) |
{
"digits_from_string": {
"type": "string",
"source": "source_name",
"function": "keep_only_digits"
}
}
<EXAMPLE INPUT>
{
"source_name": "fe223"
}
<EXAMPLE OUTPUT>
{
"digits_from_string": "223"
}
last_in_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
last_in_array | Returns the last value in an array. | array_value (type: Sequence) | default_value |
{
"last_value_in_array": {
"type": "string",
"function": "last_in_array",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [
"John",
"Bob",
"Max"
]
}
<EXAMPLE OUTPUT>
{
"last_value_in_array": "Max"
}
max_delta
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
max_delta | A directive which returns the delta between the min and max numbers in a given source array. | array_value (type: Sequence[Number]) | Number |
{
"delta_of_highest_and_lowest_values": {
"type": "numeric",
"function": "max_delta",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": [1,3,12,6,4]
}
<EXAMPLE OUTPUT>
{
"delta_of_highest_and_lowest_values": 11
}
max_value
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
max_value | Returns the max value from list of (dynamic) multiple numeric value sources. Not suitable for numerical array sources. | numeric_values (type: Sequence[Optional[Number]]) | Optional[Number] |
{
"highest_value_from_sources": {
"type": "numeric",
"function": "max_value",
"sources": [
"source_name_1",
"source_name_2",
"source_name_3"
]
}
}
{
"source_name_1": 5,
"source_name_2": 10,
"source_name_3": 7
}
{
"highest_value_from_sources": 10
}
month_from_timestamp
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
month_from_timestamp | A directive to get the month of the year of a given timestamp. | timestamp_value (type: Number) | Number |
{
"month_in_number": {
"type": "numeric",
"function": "month_from_timestamp",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": 1610569800
}
<EXAMPLE OUTPUT>
{
"month_in_number": 1
}
multiply
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
multiply | Returns the product of all the input values. If the inputs are arrays, then they have to be of the same length, and the result will be an array of that same length with all the products for each index. | input_values |
{
"product_of_multiplication": {
"type": "numeric",
"sources": [
"numeric_source_1",
"numeric_source_2"
],
"function": "multiply"
}
}
<EXAMPLE INPUT>
{
"numeric_source_1": 5,
"numeric_source_2": 10
}
<EXAMPLE OUTPUT>
{
"product_of_multiplication": 50
}
multiply_by_const
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
multiply_by_const | A directive for multiplying a value by a constant given as an arg. # TODO(adi): We assume here that zero means null, we should split this# function to multiply_by_const_no_zero. | original_value (type: Optional[Number]) | Number | multiplier |
{
"product_of_multiplication": {
"type": "numeric",
"source": "numeric_source",
"function": "multiply_by_const",
"args": [
"10"
]
}
}
<EXAMPLE INPUT>
{
"numeric_source": 5
}
<EXAMPLE OUTPUT>
{
"product_of_multiplication": 50
}
negative_check
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
negative_check | A directive to check for the negative value of the specified value. Returns True if the value is False. | boolean_value (type: bool) | Optional[bool] | treat_missing_as_false (type: bool) |
{
"value_is_negative": {
"type": "boolean",
"function": "negative_check",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": false
}
<EXAMPLE OUTPUT>
{
"value_is_negative": true
}
object_values_as_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
object_values_as_array | Returns the values of a dict as an array. Fields using this function should never be saved to crc (use "do_not_save": true in field config). TODO(itai): Create a decorator which tags such functions and doesn't allow them to be saved. | dict_value (type: Optional[Mapping]) | Sequence |
{
"array_of_values_from_object": {
"type": "string",
"function": "object_values_as_array",
"source": "source_name",
"do_not_save": true
}
}
<EXAMPLE INPUT>
{
"source_name": {
"name": "John",
"age": 30,
"city": "New York"
}
}
<EXAMPLE OUTPUT>
{
"array_of_values_from_object": [
"John",
30,
"New York"
]
}
parse_datetime_to_timestamp_seconds
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
parse_datetime_to_timestamp_seconds | A directive for parsing a datetime object into epoch timestamp seconds. If the allow_timestamps flag is set to true, the directive will not reject plain timestamps and will return a given timestamp in seconds. Can work on lists as well. | value (type: Union[str, int, Sequence[str], Sequence[int]]) | Union[Number, Sequence[Number]] | python_datetime_formats (type: Union[str, Sequence[str]]) allow_timestamps (type: bool) |
{
"timestamp_from_date": {
"type": "numeric",
"source": "datetime_object_source",
"function": "parse_datetime_to_timestamp_seconds",
"args": [
["%m/%d/%Y, %H:%M:%S", "%H:%M:%S-%m-%d-%Y"]
]
}
}
<EXAMPLE INPUT>
{
"datetime_object_source": "02/14/2021, 20:30:00"
}
<EXAMPLE OUTPUT>
{
"timestamp_from_date": 1613334600
}
<EXAMPLE INPUT>
{
"datetime_object_source": ["02/14/2021, 20:30:00", "02/18/2021, 08:30:00"]
}
<EXAMPLE OUTPUT>
{
"timestamp_from_date": [1613334600, 1613637000]
}
<EXAMPLE INPUT>
{
"datetime_object_source": "1613334600000",
"allow_timestamps": "true"
}
<EXAMPLE OUTPUT>
{
"timestamp_from_date": 1613334600
}
powerset_concat
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
powerset_concat | A directive which returns the powerset of the given array, with its values concatanated to one string for each subset. | array_value (type: Sequence) | Sequence[str] | separator (type: str) min_subset_size (type: int) max_subset_size (type: int) |
{
"stringified_concatenated_powerset": {
"type": "string",
"source": "source_name",
"function": "powerset_concat",
"args": [
", "
]
}
}
<EXAMPLE INPUT>
{
"source_name": [1,2,3]
}
<EXAMPLE OUTPUT>
{
"stringified_concatenated_powerset": [
"1.0",
"2.0",
"3.0",
"1.0, 2.0",
"1.0, 3.0",
"2.0, 3.0",
"1.0, 2.0, 3.0"
]
}
put_value_according_to_bool
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
put_value_according_to_bool | A directive which returns a value according a source boolean. | bool_value (type: bool) | use_on_false use_on_true |
{
"value_according_to_boolean": {
"type": "string",
"source": "boolean_source",
"function": "put_value_according_to_bool",
"args": [
"dogs",
"cats"
]
},
"value_according_to_boolean_2": {
"type": "string",
"source": "boolean_source_2",
"function": "put_value_according_to_bool",
"args": [
"dogs",
"cats"
]
}
}
<EXAMPLE INPUT>
{
"boolean_source": true,
"boolean_source_2": false
}
<EXAMPLE OUTPUT>
{
"value_according_to_boolean": "cats",
"value_according_to_boolean_2": "dogs"
}
range_check
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
range_check | Checks if a value is in one of the given ranges. If given an array as source input, will return a boolean response for each index in the array. | numeric_value | ranges (type: Sequence[Mapping[str, Mapping[str, Union[Number, bool]]]]) |
{
"value_is_in_range": {
"type": "boolean",
"function": "range_check",
"source": "source_name",
"args": [
[
{
"left": {
"value": 100,
"inclusive": false
}
}
]
]
}
}
<EXAMPLE INPUT>
{
"source_name": 150
}
<EXAMPLE OUTPUT>
{
"value_is_in_range": true
}
remove_if_false
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
remove_if_false | Returns first value if the boolean value is true, else None. | first_value boolean_value (type: bool) |
{
"first_value_if_true": {
"type": "numeric",
"function": "remove_if_false",
"sources": [
"source_name_1",
"source_name_2"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": 10,
"source_name_2": false
}
<EXAMPLE OUTPUT>
{
"first_value_if_true": null
}
replace_char
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
replace_char | A directive which gets a string or list of strings value, and returns it with char_to_replace replaced by char_to_insert. | original_string_or_string_list | char_to_replace char_to_insert |
{
"string_with_char_replaced_field_name": {
"type": "string",
"function": "replace_char",
"source": "source_string"
"args": "p", "l"
}
"string_list_with_char_replaced_field_name": {
"type": "string",
"function": "replace_char",
"source": "source_string_array"
"args": " ", ""
}
<EXAMPLE INPUT>
{
"source_string": "hocus pocus"
"source_string_array": ["hocus pocus", "abra cadabra"]
}
<EXAMPLE OUTPUT>
{
"string_with_char_replaced_field_name": "hucus locus"
"string_list_with_char_replaced_field_name": ["hocuspocus", "abracadabra"]
}
season_from_timestamp
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
season_from_timestamp | A directive to get the season of the year of a given timestamp. | timestamp_value (type: Number) | is_northern_hemisphere |
{
"season_of_timestamp_north": {
"type": "string",
"function": "season_from_timestamp",
"source": "source_name",
"args": [
true
]
},
"season_of_timestamp_south": {
"type": "string",
"function": "season_from_timestamp",
"source": "source_name",
"args": [
false
]
}
}
<EXAMPLE INPUT>
{
"source_name": 1610569800
}
<EXAMPLE OUTPUT>
{
"season_of_timestamp_north": "winter",
"season_of_timestamp_south": "summer"
}
shannon_entropy
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
shannon_entropy | Returns the shannon entropy of the values in the given array. the entropy is calculated as S = -sum(array_value * log(array_value), axis=axis). This routine will normalize the values if they don’t sum to 1. When use_base_2=false, the log base is e. | array_value | use_base_2 |
{
"array_shannon_entropy": {
"type": "numeric",
"function": "shannon_entropy",
"source": "array_source_name"
}
},
{
"array_shannon_entropy_base_2": {
"type": "numeric",
"function": "shannon_entropy",
"source": "array_source_name",
"args":[
true
]
}
}
<EXAMPLE INPUT>
{
"array_source_name": [1/2,1/2]
}
<EXAMPLE OUTPUT>
{
"array_shannon_entropy": 0.6931471805599453
},
{
"array_shannon_entropy_base_2": 1
}
sort_and_concat_mapper
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
sort_and_concat_mapper | A mapper directive which, for every (sequence) element in the source array, sorts and concats as string using the given separator string. | array_value (type: Sequence[Sequence]) | Sequence | separator |
{
"mapped_and_sorted_arrays": {
"type": "string",
"function": "sort_and_concat_mapper",
"source": "source_name",
"args": [
", "
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
[
"John",
"Bob",
"Sharon",
"Joe"
],
[
3,
2,
6,
4
],
[
"dog",
"cat",
"monkey"
]
]
}
<EXAMPLE OUTPUT>
{
"mapped_and_sorted_arrays": [
[
"Bob, Joe, John, Sharon"
],
[
"2, 3, 4, 6"
],
[
"cat, dog, monkey"
]
]
}
sort_array_of_objects_by_key
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
sort_array_of_objects_by_key | Returns the given array, sorted by the sort_key. If there a none value under sort_key, we remove it before sorting. | array_value (type: Sequence) | Optional[Sequence] | sort_key (type: str) reverse |
{
"sorted_array": {
"function": "sort_array_of_objects_by_key",
"source": "source_name",
"do_not_save": true,
"args": [
"age"
]
},
"first_in_sorted_array": {
"function": "first_in_array",
"source": "sorted_array",
"do_not_save": true
},
"age_of_first_in_sorted_array": {
"type": "numeric",
"function": "field_value_from_object",
"source": "first_in_sorted_array",
"args": [
"age"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 20
},
{
"name": "Bob",
"age": 30
},
{
"name": "Max",
"age": 19
}
]
}
<EXAMPLE OUTPUT>
{
"age_of_first_in_sorted_array": "19"
}
split_string_to_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
split_string_to_array | A directive to split a string into an array. | string_value (type: str) | Sequence[str] | separator (type: str) array_wrapping_char (type: str) inner_string_wrapping_char (type: str) |
{
"array_of_strings_after_split": {
"type": "string",
"function": "split_string_to_array",
"source": "string_source_name",
"args": [
", "
]
}
}
{
"string_source_name": "cat, dog, frog"
}
{
"array_of_strings_after_split": ["cat","dog","frog"]
}
sub_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
sub_array | Returns a sub array for the given array | array_value (type: Sequence) | Optional[Sequence] | start_index (type: int) end_index (type: int) default_value |
{
"sub_array_according_to_args": {
"type": "string",
"function": "sub_array",
"source": "source_name",
"args": [
2,
5
]
}
}
<EXAMPLE INPUT>
{
"source_name": [1,2,3,4,5,6,7,8]
}
<EXAMPLE OUTPUT>
{
"sub_array_according_to_args": [3,4,5,6]
}
sum_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
sum_array | Returns the sum of all values in the given array. | array_value |
{
"sum_of_array": {
"type": "numeric",
"function": "sum_array",
"source": "array_source_name"
}
}
<EXAMPLE INPUT>
{
"array_source_name": [1,3,4]
}
<EXAMPLE OUTPUT>
{
"sum_of_array": 8
}
sum_field_in_array
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
sum_field_in_array | A directive which returns the sum of the values of a key in an array of dict objects. | array_value (type: Sequence[Mapping[str, Number]]) | Number | key (type: str) |
{
"sum_of_values_by_key": {
"type": "numeric",
"function": "sum_field_in_array",
"source": "source_name",
"args": [
"age"
]
}
}
<EXAMPLE INPUT>
{
"source_name": [
{
"name": "John",
"age": 13
},
{
"name": "Bob",
"age": 20
},
{
"name": "Sharon",
"age": 17
}
]
}
<EXAMPLE OUTPUT>
{
"sum_of_values_by_key": 50
}
sum_formula
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
sum_formula | A directive for summing (dynamic) multiple numeric value sources. Not suitable for numerical array sources. | numeric_values (type: Sequence[Optional[Number]]) | Number |
{
"sum_of_formula_sources": {
"type": "numeric",
"function": "sum_formula",
"sources": [
"source_name_1",
"source_name_2",
"source_name_3"
]
}
}
<EXAMPLE INPUT>
{
"source_name_1": 5,
"source_name_2": 6,
"source_name_3": 10
}
<EXAMPLE OUTPUT>
{
"sum_of_formula_sources": 21
}
timestamp_delta_days
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
timestamp_delta_days | A directive for calculating days delta of two timestamps (provided in seconds/millis). | first_timestamp (type: Optional[Number]) second_timestamp (type: Optional[Number]) | Optional[Number] | default_value |
{
"delta_of_timestamps_in_days": {
"type": "numeric",
"sources": [
"primary_source",
"secondary_source"
],
"function": "timestamp_delta_days"
}
}
<EXAMPLE INPUT>
{
"primary_source": 1613334600,
"secondary_source": 1613248200
}
<EXAMPLE OUTPUT>
{
"delta_of_timestamps_in_days": 1
}
to_int_string
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
to_int_string | A directive which gets a numeric (or numeric-convertible) value, and returns its string representation, rounded, without decimal point. | original_value | str |
{
"numeric_to_int_field_name": {
"type": "string",
"function": "to_int_string",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": 4.55
}
<EXAMPLE OUTPUT>
{
"numeric_to_int_field_name": "4"
}
use_value_mapping
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
use_value_mapping | A directive which returns the mapped value of a key value. | key_value (type: str) | mapper (type: str) |
{
"fields": {
"mapped_value_of_key": {
"type": "string",
"function": "use_value_mapping",
"source": "key_source_name",
"args": [
"mapper_data_name"
]
}
},
"data": {
"second_set": {
"A": "Dogs",
"B": "Cats",
"F": "Monkeys",
"D": "Cows",
"E": "Birds"
}
}
}
<EXAMPLE INPUT>
{
"key_source_name": "B"
}
<EXAMPLE OUTPUT>
{
"mapped_value_of_key": "Cats"
}
zip_to_us_state
Function | Description | Sources | Output | Arguments |
---|---|---|---|---|
zip_to_us_state | A directive which gets a US zip code value and returns the US state code it belongs to. If the zip not found (or given an invalid value) - returns INVALID string. | string_or_int_value (type: Union[str, int]) | str |
{
"zip_code_to_state_field_name": {
"type": "string",
"function": "zip_to_us_state",
"source": "source_name"
}
}
<EXAMPLE INPUT>
{
"source_name": 90401
}
<EXAMPLE OUTPUT>
{
"zip_code_to_state_field_name": "CA"
}
Updated about 2 months ago