Resource directives describe a resource, provide instructions how to load them and assign the resource an identifier such that it can be referenced and loaded. In the job definitions they are used to load resources upfront that are big or repeatedly accessed, such as judgement lists.
Note: the supplier option VALUES_FROM_NODE_STORAGE
is usually not valid since it assumes that a resource with the defined
identifier has already been loaded when this resource is accessed, which might not be the case.
Resource Directive Types | |
---|---|
JUDGEMENT_PROVIDER | Judgement provider resource. A judgement provider is used to retrieve judgement values for a query-productId pair. |
MAP_STRING_TO_DOUBLE_VALUE | Map[String, Double] resource. |
MAP_STRING_TO_STRING_VALUES | Map[String, List[String]] resource. |
_STRING_VALUES _ | List[String] resource. |
Let’s walk over the provided load options per type.
Load Option | |
---|---|
JUDGEMENTS_FROM_FILE | Provide a file path (relative to the configured base path) from which judgements are loaded. Note that the format (CSV or JSON_LINES ) depends on your configuration (see docker-compose documentation). For formats of the different types, refer to the file-formats section of the documentation. |
VALUES_FROM_NODE_STORAGE | Provide identifier that refers to a resource of type JUDGEMENT_PROVIDER . Not recommended, since this would assume the resource this resource refers to would need to be already loaded when this resource is accessed. |
The options here are restricted right now and thus right now should not be used. Other ways of defining resources of this type are planned.
Load Option | |
---|---|
VALUES_FROM_NODE_STORAGE | Provide identifier that refers to a resource of type MAP_STRING_TO_DOUBLE_VALUE . Not recommended, since this would assume the resource this resource refers to would need to be already loaded when this resource is accessed. |
Load Option | |
---|---|
JSON_VALUES_MAPPING_TYPE | Selecting this will provide the option to enter key values and assign one or multiple values to it. Thus this option means manual configuration of the key-value assignments. |
JSON_VALUES_FILES_MAPPING_TYPE | Selecting this will provide the option to enter (key, value) pairs, where the value for each key (only single value) gives the path to a file that contains one single value per line (file path relative to configured base path). Thus on creation of the resource, all lines of the referenced files for each key will be read to compose all values for all keys. |
JSON_SINGLE_MAPPINGS_TYPE | Set a file path (relative to the configured base path) to a json file containing single mappings, that is each key maps to a plain string value. |
JSON_ARRAY_MAPPINGS_TYPE | Same as JSON_SINGLE_MAPPINGS_TYPE , but instead of plain string values contains a List of values per key. |
FILE_PREFIX_TO_FILE_LINES_TYPE | Given a directory path (relative to the defined base path) and a file suffix, use the file names with suffix removed as keys and each line of the respective file as value. |
CSV_MAPPING_TYPE | Given a file path (relative to configured base path), specify a csv file, the column delimiter, the index of the key column and the index of the value column. Will assign a List of values to each key value. Thus, if multiple rows assign a value to the same key, all those values will appear in the assigned values for that key. |
VALUES_FROM_NODE_STORAGE | Provide identifier that refers to a resource of type MAP_STRING_TO_STRING_VALUE . Not recommended, since this would assume the resource this resource refers to would need to be already loaded when this resource is accessed. |
Example for supplier
config for JSON_VALUES_FILES_MAPPING_TYPE
:
{
"type": "JSON_VALUES_FILES_MAPPING_TYPE",
"values": {
"p1": "data/parameters/values1.txt",
"p2": "data/parameters/values2.txt"
}
}
Example for supplier
config for JSON_SINGLE_MAPPINGS_TYPE
:
{
"type": "JSON_SINGLE_MAPPINGS_TYPE",
"values": "data/parameters/singleValueMapping.json"
}
Example for supplier
config for JSON_ARRAY_MAPPINGS_TYPE
:
{
"type": "JSON_ARRAY_MAPPINGS_TYPE",
"values": "data/parameters/multiValueMapping.json"
}
Example for supplier
config for FILE_PREFIX_TO_FILE_LINES_TYPE
:
{
"type": "FILE_PREFIX_TO_FILE_LINES_TYPE",
"directory": "data/parameters/fileNameToValues",
"files_suffix": ".txt"
}
Example for supplier
config for CSV_MAPPING_TYPE
:
{
"type": "CSV_MAPPING_TYPE",
"values": "data/parameters/mapping.csv",
"column_delimiter": "\\t",
"key_column_index": 0,
"value_column_index": 1
}
Define a sequence of values.
Load Option | |
---|---|
FROM_ORDERED_VALUES_TYPE | Contains a range of options to define the data. For details see table below. |
PARAMETER_VALUES_TYPE | Manually add each value. |
VALUES_FROM_NODE_STORAGE | Provide identifier that refers to a resource of type MAP_STRING_TO_DOUBLE_VALUE . Not recommended, since this would assume the resource this resource refers to would need to be already loaded when this resource is accessed. |
FROM_ORDERED_VALUES_TYPE sources | |
---|---|
FROM_FILENAME_KEYS_TYPE | Given a directory path (relative to the defined base path) and a file suffix, use the file names with removed suffix as values. |
FROM_FILES_LINES_TYPE | Specify a file path (relative to the configured base path) from which each line is picked as a single value. |
FROM_VALUES_TYPE | Manually define a sequence of values, same as the PARAMETER_VALUES_TYPE option above. |
FROM_RANGE_TYPE | Define start, end and stepSize values. This will generate a sequence of floating point numbers. NOTE: in case you intend to use this as a *_REPLACE parameter (see request parameter section), take into account that in string replace a value would appear as floating point, not as integer. |