.. _metadata-label: Metadata ================ The **Phenome** is made of up a large amount of **metadata**, currently represented by collections of files in JSON format. These JSON files can live in a couple different places: - /{app_dir}/{**app_name**}/config/meta (application specific location) - /{app_dir}/**phenome**/config/meta (phenome default location) The aim of the Phenome is to be able to contain enough detail to accurately represent an object and environment. The idea is that with better information comes better results in terms of decision making and predictive outcomes. File Structure ----------------- The entire JSON Phenome File Metadata is separated into several sections: +---------------+------------------------------------+ | SECTION | DESCRIPTION | +===============+====================================+ | HEADER | Information about the JSON file | +---------------+------------------------------------+ | ENUMS | Defines Custom ENUMS | +---------------+------------------------------------+ | UI_MODELS | Defines Models for the UI | +---------------+------------------------------------+ | OBJECT_MODELS | Defines the rest of the Phenome | +---------------+------------------------------------+ The OBJECT_MODELS section is broken up into a series of one or more OBJECT_MODEL sections. Inside each OBJECT_MODEL section consists of a HEADER section and a PHENOME section. The following JSON structure represents a Phenome JSON file: .. code-block:: json { "header_field_1": "header_field_1_value", "header_field_2": "header_field_2_value", "enums": { "enum": [ { }, { } ] }, "ui_models": { "ui_model": [ { }, { } ] }, "object_models": { "object_model": [ { "header_field_1": "header_field_1_value", "header_field_2": "header_field_2_value", "phenome": { "property_model": [ { }, { } ], "action_model": [ { }, { } ], "data_model": [ { }, { } ], "relation_model": [ { }, { } ], "thought_model": [ { }, { } ] }, "properties": { "property": [ { }, { } ] } } ] } } Header Section ------------------- Example of a JSON file HEADER section: :: "file_description": "Phenome system configuration", "phenome_app_id": 0, "phenome_app_name": "phenome", "phenome_app_title": "my_application", "phenome_app_version": "0.0.1", "phenome_app_description": "My First Phenome Application", "phenome_app_classname": "myapp.startupclass", "phenome_developer_id": 0 - **phenome_app_classname** defines the Flask Class that will be initiated on startup - **phenome_developer_id** defines which unique developer created the app - **phenome_app_id** defines a unique ID for the APP For now, both IDs can be set to 0. If running more than one APP at the same time (i.e. if you have defined more than one APP in metadata JSON files), then these numbers must be unique. The IDs will both be set if the Metadata definitions are created and/or stored in the Phenome cloud, and will be associated with a developer account. ENUM Section ----------------- This section allows the developer to create a unique set of customizable ENUMs for use in the code or by the other metadata. Example of an ENUM definition: .. code-block:: json { "id": "_STATE_TYPES_", "description": "State Types for object model properties.", "values": [ { "key": "UNKNOWN", "value": "0", "attributes": [{ "description": "Object State is 'UNKNOWN'"}] }, { "key": "OFF", "value": "1", "attributes": [{ "description": "Object State is 'OFF'"}] }, { "key": "ON", "value": "2", "attributes": [{ "description": "Object State is 'ON'"}] }, { "key": "CYCLE", "value": "3", "attributes": [{ "description": "Object State is 'CYCLE'"}] } ] } This **_STATE_TYPES_** ENUM defines 4 specific states for an object's property, if defined as a _STATE_TYPE_ property. UI Model ----------------- This section allows for some re-use of UI layout and actions in the embedded UI. Very often something defined in the ActionModel should behave a certain way and having a standard way to interact with the user, without duplication, is convenient. If not using the agent UI and only accessing the agent via the API, this section is not relevant. Example of a UI_MODEL definition: .. code-block:: json { "id": "power_outlet_toggle_button", "type": "toggle_button", "title": "Power Outlet {outlet}", "action-on": "power_on_outlets({outlet})", "action-off": "power_off_outlets({outlet})", "data-on": "ON", "data-off": "OFF", "data-onstyle": "success", "data-offstyle": "danger", "alter-disabled-style": true } This shared UI_MODEL **power_outlet_toggle_button** allows the UI to display a Power Toggle button. When clicked, it will execute either the **power_on_outlets** or **power_off_outlets** command on that object for the specified {outlet} reference. ObjectModel ------------ For an overview of how the ObjectModel works in action, please see :ref:`objectmodeloverview` Example of OBJECT_MODEL section HEADER: :: "id": "ROOT_REFRIGERATOR", "description": "All Refrigerators", "model_classtype": "APPLIANCE", "model_classname": "appliancedoctor.appliances.fridges.basefridge.BaseFridge", "model_classname_poller": "APPLIANCE_POLLER", "model_classname_results": "appliancedoctor.appliance_results.ApplianceResults", "model_classname_processor": "appliancedoctor.appliance_processor.ApplianceProcessor", - **id** is the unique OBJECT MODEL ID for this object - **description** - describes the Objects of this type of Object Model - **model_classtype** describes what general type of object this is - **model_classname** lists the code definition that will be created to represent an instance of this object - **model_classname_poller** will tell the system to dynamically create an "appliance" poller on startup - **model_classname_results** the Results collection object used to store the data for this type of object - **model_classname_processor** the Poller Processor used to process the results The rest of an Object Model object is comprised of a **phenome**: - :ref:`propertymodel-label` - :ref:`datamodel-label` - :ref:`actionmodel-label` - :ref:`relationmodel-label` - :ref:`thoughtmodel-label` .. _propertymodel-label: PropertyModel ----------------- Any and all properties of a particular type of object are defined here. A Property Model is defined on a ROOT Object type, and Properties are defined for an object based on that type. Properties are added directly to objects and can be accessed directly. - **property_type** - can be defined as **STATE** or **CONFIGURATION** - **value_type** - TEXT, INTEGER, REAL, ENUM - **default** - The default value for all properties using that property model A **PropertyModel** is typically defined on a ROOT Object type, which has default values on initialization, but can also be applied to subtypes. Specific **Properties** can be defined on child objects based on the ROOT type, overriding default values or behaviors. Properties can also be **nulled** for child objects so it seems like that property was never defined for that object in the first place. Example of a Property Model definition: .. code-block:: json { "name": "power_state", "description": "Powered state of the object", "property_type": "STATE", "value_type": "INTEGER", "default": 1, "attributes": [{ "show_in_ui": "false", "required": 0, "value_enum":"_STATE_TYPES_", "valid_values":"OFF,ON"}] } This property **power_state** allows an object to store a powered state of ENUM value **_STATE_TYPES_**, and specifies that the only valid values are **ON** and **OFF**. .. _datamodel-label: DataModel ----------------- The Data Model defines all interesting KPIs that will be collected by the system during a poll cycle for a particular object. It describes the storage type for the KPI and the handling associated with it. Example of a Data Model definition: .. code-block:: json { "name": "temperature", "description": "temperature", "attributes": [{ "value_type": "float64", "units": "celcius", "aggr": "mean"}] } This datamodel entry **temperature** is defined as a float and is stored in units of degrees Celcius, and when aggregated over time, the system will use the mean. .. _actionmodel-label: ActionModel ----------------- Actions belonging to objects are defined in this section. Two basic types of actions that can be defined are: - **methods** that will become callable from other objects and the rest of the system - **sensorchecks** that will be executed during each poll cycle Example of a **method** definition: .. code-block:: json { "name": "make_ice", "description": "Tells the Fridge to Make Ice", "attributes": [{ "classname": "self", "methodname":"make_ice", "show_in_ui": "true", "delay_secs": "60", "create_method": "True"}] } This section tells the system that there is a callable action named **"make_ice"** on this object and allows it to be called in the UI and through the API. Please note that of course there must also be a method "make_ice()" in the code that will actually tell the refrigerator to make the ice. Example of a **sensorcheck** definition: .. code-block:: json { "name": "check_ice", "description": "Checks if the fridge is out of ice", "attributes": [{ "classname": "phenome.extensions.sensorchecks.generic_sensorcheck.GenericCheck", "methodname":"execute", "input":"object.ice_level", "warning_level": "object.low_ice_count_threshold", "has_warning":"warning_level is not None and warning_level > 0 and object.ice_level < warning_level", "warning_message":"'Fridge is low on ice ({}<{})'.format(_result_avg, object.low_ice_count_threshold)", "behavior_on_warning":["_notify","make_ice"] }] } This section is executed after each poll cycle (typically 1 minute). If the fridge is low on ice, it will send a notification to the user and tell the fridge to make some more ice. .. _relationmodel-label: RelationModel ----------------- The relation model allows for descriptions of **dependency** relationships between objects and allows actions to be defined according to various object states and relationships. The model is primarily based on the concept of relationship types (see the ENUM **_RELATION_TYPES_**). Objects can easily be "related" to other objects by specifying a **relation_type** and some details. Example of a **dependency** definition: .. code-block:: json { "name": "temp_sensor_dependency", "description": "dependency on another object's temperature sensor values", "attributes": [{ "classname": "phenome.extensions.dependencychecks.generic_dependencycheck.GenericDependencyCheck", "relation_type": "REMOTE_SENSOR", "relation_hasattr": "temperature", "related_model":"ANY", "repeat_delay_secs": "120", "enabled": "true", "notify": "true", "test_for_state_1": "relation.temperature > object.temp_error and object.power_state == 2", "test_for_state_2": "relation.temperature < object.temp_warning and object.power_state != 2", "behavior_on_state_1": "object.power_off()", "behavior_on_state_2": "object.power_on()", "message_on_state_1": "'Powering OFF ({}) due to high temperature ({}) reported by ({})'.format(object.unique_id,relation.temperature,relation.unique_id)", "message_on_state_2": "'Powering ON ({}) due to lower temperature ({}) reported by ({})'.format(object.unique_id,relation.temperature,relation.unique_id)", "state_1_is_error": "true", "state_2_is_error": "false" }] } This particular dependency definition describes a **REMOTE_SENSOR** relationship between two objects and is concerned with the temperature of the remote sensor. If the sensor senses a high temperature that crosses a threshold, the system will power off the object. If the object is already powered off, and the temperature is below a certain threshold, then the object will be powered on. In both cases, a notification will be sent. In addition, it will wait at least 2 minutes between any powerOFF or powerON events. .. _thoughtmodel-label: ThoughtModel ----------------- The thought model is used to specify details for prediction problem definitions. The prediction engine uses this information to try to come up with a best performing Machine Learning Model. Example of a **prediction problem** definition to predict future temperatures: .. code-block:: json { "name": "temperature", "description": "predict future temperature", "attributes": [ { "algo": "AUTO", "data_model": [{"data_in": "temperature,fan_status,chip_status,hashrate", "data_out": "temperature"}], "period_minutes": [{"steps_in": 6, "steps_out": 1}], "seasonality": [{"include_time": "true", "include_date": "false"}] } ] } - **algo** describes the algorithm to use (AUTO, CNN, MLP, LSTM) - **data_model.data_in** specifies the data used to feed into the models - **data_model.data_out** specifies the desired prediction KPI - **period_minutes** describes the number of timesteps passed in, and the desired number of timesteps to be predicted - **seasonality** allows time and date to be used as inputs into the model