To visualize the schemas you can copy paste them to http://chris.photobooks.com/json/

collectd json schema

This schema describes the collectd internal types. This schema will be referenced from other mapping schema.

Ceilometer use case

Ceilometer old samples schema

Code Block

language	bash
linenumbers	true

{
	"$schema{
    "counter_name": "http://json-schema.org/draft-04/schema#instance",
	"title    "counter_type": "gauge"collectd,
  Base Schema", 	"descriptioncounter_unit": "instance"A,
mapping from collectd meters and events to *other* meters and events",
	"type": "object",
	"properties": {
		"value list": {
			"description": "Statistics in collectd consist of a value list",
			"type": "object",
			"properties": {
				"values" : {
					"type": "array",
					"anyOf" : [ 
						{ "type" : "object", "properties" : { "absolute" : { "type" : "number" } } },
						{ "type" : "object", "properties" : { "counter" : { "type" : "number" } } },
						{ "type" : "object", "properties" : { "derive" : { "type" : "number" } } },
						{ "type" : "object", "properties" : { "gauge" : { "type" : "number" } } }
					]
				},
				"value length" : {
					"description": "The number of values in the data set",
					"type": "number"
				},
				"time" : {
					"description": "Time stamp at which the value was collected",
					"type": "number"
				},
				"Interval" : {
					"description": "interval at which to expect a new value",
					"type": "number"
				},
				"host" : {
					"description": "used to identify the host",
					"type": "string"
				},
				"plugin" : {
					"description": "used to identify the plugin",
					"type": "string"
				},
				"plugin instance" : {
					"description": "used to group a set of values together",
					"type": "string"
				},
				"type" : {
					"description": "unit used to measure a value",
					"type": "string"
				},
				"type instance " : {
					"description": "used to distinguish between values that have an identical type",
					"type": "string"
				},
				"metadata" : {
					"description": "an opaque data structure that enables the passing of additional information about a value list",
					"type": "string"
				}
			}
		},
		"notifications": {
			"description": "Notifications in collectd are generic messages",
			"type": "object",
			"properties": {
				"severity" : {
						"description": "can be one of OKAY, WARNING, and FAILURE",
						"type": "string"
				},
				"time" : {
					"description": "Time stamp at which the event was collected",
					"type": "number"
				},
				"message" : {
					"description": "The notification message",
					"type": "string"
				},
				"host" : {
					"description": "used to identify the host",
					"type": "string"
				},
				"plugin "counter_volume": 1.0,
    "message_id": "5460acce-4fd6-480d-ab18-9735ec7b1996",
    "project_id": "35b17138-b364-4e6a-a131-8f3099c5be68",
    "recorded_at": "2015-01-01T12:00:00",
    "resource_id": "bd9431c1-8d69-4ad3-803a-8d4a6b89fd36",
    "resource_metadata": {
        "name1": "value1",
        "name2": "value2"
    },
    "source": "openstack",
    "timestamp": "2015-01-01T12:00:00",
    "user_id": "efd87807-12d2-4b38-9c70-5f5c2ac427ff"
}

Currently data normalization or mappings are managed manually in the ceilometer write plugin:

A snippet from https://github.com/openstack/collectd-ceilometer-plugin/blob/master/collectd_ceilometer/units.py:

Code Block

language	py
linenumbers	true

# Unit mappings in alphabetical order

UNITS = {

    'apache.apache_idle_workers':   'Workers',

    'apache.apache_bytes':          'B/s',

    'apache.apache_requests':       'Req/s',

    'apache.apache_scoreboard':     'Slots',

    'apache.apache_connections':    'Connections',




    'apcups.timeleft':      's',

    'apcups.temperature':   '°C',

    'apcups.percent':       'Load',

    'apcups.charge':        'Ah',

    'apcups.frequency':     'Hz',

    'apcups.voltage':       'V',


...

and the meter code itself can be found here: https://github.com/openstack/collectd-ceilometer-plugin/blob/master/collectd_ceilometer/meters/base.py fields are mapped as follows:

ceilometer meter field	collectd field
counter_name	plugin, type
counter_type	"gauge"-> "gauge", "derive"-> "delta "absolute"-> "cumulative" "counter"-> "cumulative"
counter_unit	unit
counter_volume	value
message_id	-
project_id	-
recorded_at	-
resource_id	hostname, plugin_instance, type_instance
resource_metadata	meta
source	collectd
timestamp	time
user_id	-

Proposal 1

collectd json schema

This schema describes the collectd internal types. This schema will be referenced from other mapping schema.

Code Block

language	bash
linenumbers	true

{
	"$schema": "http://json-schema.org/draft-04/schema#",
	"title": "collectd Base Schema",
	"description": "A mapping from collectd meters and events to *other* meters and events",
	"type": "object",
	"properties": {
		"value list": {
			"description": "Statistics in collectd consist of a value list",
			"type": "object",
			"properties": {
				"values" : {
					"descriptiontype": "used to identify the pluginarray",
					"typeanyOf" : "string"[ 
				},
				"plugin instance{ "type" : "object", "properties" : { 					"descriptionabsolute" : { "type"used to: group"number" a} set of values together"} },
						{ "type" : "string"
				},
				"typeobject", "properties" : { "counter" : { 					"descriptiontype" : "number"unit used} to measure a value"} },
						{ "type" : "string"
				}object", 				"type instance "properties" : { 					"descriptionderive" : { "used to distinguish between values that have an identical type"type" : "number" } } },
						{ "type" : "string"
				}object", 				"metadataproperties" : { 					"descriptiongauge": "an opaque data structure that enables the passing of additional information about a value list", : { "type" : "number" } } }
					"type": "string"
]
				},
				}
		}
	}
}

collectd to ceilometer mapping example

I think 2 things are required. The message format schema for the end point and the mapping schema.

Former user (Deleted) will examine 1 combined schema for next week

...

Ceilometer old samples schema

Code Block

language	bash

{
    "counter_name": "instance",
    "counter_type": "gauge",
    "counter_unit": "instance",
    "counter_volume": 1.0,
    "message_id": "5460acce-4fd6-480d-ab18-9735ec7b1996",
    "project_id": "35b17138-b364-4e6a-a131-8f3099c5be68",
    "recorded_at": "2015-01-01T12:00:00",
    "resource_id": "bd9431c1-8d69-4ad3-803a-8d4a6b89fd36",
    "resource_metadata": {
        "name1": "value1",
        "name2": "value2"
    },
    "source": "openstack",
    "timestamp": "2015-01-01T12:00:00",
    "user_id": "efd87807-12d2-4b38-9c70-5f5c2ac427ff"
}

Currently data normalization or mappings are managed manually in the ceilometer write plugin:

A snippet from https://github.com/openstack/collectd-ceilometer-plugin/blob/master/collectd_ceilometer/units.py:

Code Block

language	py
linenumbers	true

# Unit mappings in alphabetical order

UNITS = {

    'apache.apache_idle_workers':   'Workers',

    'apache.apache_bytes':          'B/s',

    'apache.apache_requests':       'Req/s',

    'apache.apache_scoreboard':     'Slots',

    'apache.apache_connections':    'Connections',




    'apcups.timeleft':      's',

    'apcups.temperature':   '°C',

    'apcups.percent':       'Load',

    'apcups.charge':        'Ah',

    'apcups.frequency':     'Hz',

    'apcups.voltage':       'V',


...

and the meter code itself can be found here: https://github.com/openstack/collectd-ceilometer-plugin/blob/master/collectd_ceilometer/meters/base.py fields are mapped as follows:

ceilometer meter field	collectd field
counter_name	plugin, type
counter_type	"gauge"-> "gauge", "derive"-> "delta "absolute"-> "cumulative" "counter"-> "cumulative"
counter_unit	unit
counter_volume	value
message_id	-
project_id	-
recorded_at	-
resource_id	hostname, plugin_instance, type_instance
resource_metadata	meta
source	collectd
timestamp	time
user_id	-

...

"value length" : {
					"description": "The number of values in the data set",
					"type": "number"
				},
				"time" : {
					"description": "Time stamp at which the value was collected",
					"type": "number"
				},
				"Interval" : {
					"description": "interval at which to expect a new value",
					"type": "number"
				},
				"host" : {
					"description": "used to identify the host",
					"type": "string"
				},
				"plugin" : {
					"description": "used to identify the plugin",
					"type": "string"
				},
				"plugin instance" : {
					"description": "used to group a set of values together",
					"type": "string"
				},
				"type" : {
					"description": "unit used to measure a value",
					"type": "string"
				},
				"type instance " : {
					"description": "used to distinguish between values that have an identical type",
					"type": "string"
				},
				"metadata" : {
					"description": "an opaque data structure that enables the passing of additional information about a value list",
					"type": "string"
				}
			}
		},
		"notifications": {
			"description": "Notifications in collectd are generic messages",
			"type": "object",
			"properties": {
				"severity" : {
						"description": "can be one of OKAY, WARNING, and FAILURE",
						"type": "string"
				},
				"time" : {
					"description": "Time stamp at which the event was collected",
					"type": "number"
				},
				"message" : {
					"description": "The notification message",
					"type": "string"
				},
				"host" : {
					"description": "used to identify the host",
					"type": "string"
				},
				"plugin" : {
					"description": "used to identify the plugin",
					"type": "string"
				},
				"plugin instance" : {
					"description": "used to group a set of values together",
					"type": "string"
				},
				"type" : {
					"description": "unit used to measure a value",
					"type": "string"
				},
				"type instance " : {
					"description": "used to distinguish between values that have an identical type",
					"type": "string"
				},
				"metadata" : {
					"description": "an opaque data structure that enables the passing of additional information about a value list",
					"type": "string"
				}
			}
		}
	}
}

collectd to ceilometer mapping example

I think 2 things are required. The message format schema for the end point and the mapping schema.

Former user (Deleted) will examine 1 combined schema for next week

...

Issues:

Proposal 1 is too highlevel and doesn't get into the internals of what meters get mapped to what and what units should be associated with the meters/events.

Proposal 2

collectd simple mapping schema (note this is too clunky so it can be ignored):

...

Version	Old Version 10	New Version 11
Changes made by	Former user (Deleted)	Former user (Deleted)
Saved on	Nov 22, 2016	Nov 23, 2016

Versions Compared

Key

collectd json schema

Ceilometer use case

Proposal 1

collectd json schema

collectd to ceilometer mapping example

collectd to ceilometer mapping example

Issues:

Proposal 2

collectd simple mapping schema (note this is too clunky so it can be ignored):

Page Comparison

Versions Compared

Key

collectd json schema

Ceilometer use case

Proposal 1

collectd json schema

collectd to ceilometer mapping example

collectd to ceilometer mapping example

Issues:

Proposal 2

collectd simple mapping schema (note this is too clunky so it can be ignored):