#Custom REST API labels

1 messages · Page 1 of 1 (latest)

lyric violet
#

We currently use ZAPI and ZAPIPERF for harvest and want to move to using REST.

Under zapi, if I wanted to add another label, I duplicate the yml of the one I want to modify(making a yml_custom.yml) and only have the list of changes I want to add and then it 'did the right thing' and appended it to the default file when harvest started up.

When I attempt this with REST yml files, this seems to not work anymore (at least from what I am seeing).

Is there some sort of config I am missing in the migration to REST that will allow this behavior? I also tried to modify the existing files but all that did is leave the original label BLANK and populated my new one.

Example:

conf/rest/9.12.0/aggr.yml:

counters:

  • ^name => aggr
  • ^name => aggregate

export_options:
instance_keys:
- aggr
- aggregate

This shows the addition I made and the result was:

aggr="",aggregate="<insert actual aggr here>"

Clearly I am missing something.

#

in ZAPI I simply did this:

aggr_custom.yaml:
name: Aggregate
query: aggr-get-iter
object: aggr

counters:
aggr-attributes:
- ^aggregate-name => aggregate

export_options:
instance_keys:
- aggregate

The result from this was same as the REST one but BOTH aggr and aggregate were populated.

smoky grove
lyric violet
#

will it ever? That is such a nice feature to be able to track the changes made in the current zapi/zapiperf because the file only has the changes in it

#

ok, so if I modify the above rest file "aggr.yml" why doesn't the original label keep the same data as the new label I created, why is it blank?

smoky grove
#

Rest templates has an endpoints DSL, which ZAPI doesn’t. We thought merging them would make it complex to track multiple parts from a usability point of view.
I think only the last will take precedence probably. Let me check.

lyric violet
#

what if I need both of them, just like what ZAPI/ZAPIPERF did when I added them?

#

that seems strange to offer it in one part of the code but not the other

smoky grove
#

See if something like below works


name:               Aggregate
query:              api/storage/aggregates
object:             aggr

counters:
  - ^^uuid                                                         => uuid
  - ^name                                                          => aggr
  - ^state                                                         => state
  - space.block_storage.size                                       => space_total

plugins:
  - LabelAgent:
      join:
        - aggregate `` aggr

export_options:
  instance_keys:
    - aggr
    - aggregate
#

aggr_space_total{datacenter="u2",cluster="umeng-aff300-01-02",aggr="umeng_aff300_aggr2",aggregate="umeng_aff300_aggr2"} 8568893079552
aggr_space_total{datacenter="u2",cluster="umeng-aff300-01-02",aggr="test1",aggregate="test1"} 6855114465280

#

This is just an example. Please add keys or export options as needed.

lyric violet
#

if that worked, that would make it much clearer change. I'll give it a try

smoky grove
#

Yes LabelAgent plugin does this.

lyric violet
#

that worked

smoky grove
#

Great. Thanks!

lyric violet
#

simply join and instance key and it is added

#

might be a good idea to add that to documentation somewhere as a example for adding a duplicate of another label

smoky grove
#

Sure will do and share with you

lyric violet
#

thanks

smoky grove
lyric violet
#

thanks again

lyric violet
#

One other thing came up. If I want to add a new object / metric into harvest that doesn't exist, can I set the objects: labels in default.yaml to anything I want?

Example:

Let's say I want to add a yaml for querying kites (fictional item in netapp REST API), can I just do this?

objects:

Kites:           Kites.yaml
#

or do the names like "Kites:" have to be defined in the upstream code to be usable?

terse fiber
#

hi @lyric violet those names do not need to exist beforehand. If you add that to your conf/rest/default.yaml file and the Kites.yaml file exists and is a valid template, it will be used at runtime

lyric violet
#

awesome.

#

thanks

terse fiber
#

you're welcome

lyric violet
#

should I be using a custom.yaml instead of modifying the default.yaml? I normally would just add them but given the restriction on REST yaml (not merged, just need to duplicate and modify) what would make sense for rest? Simply modifying the default or ????

smoky grove
#

custom.yaml is better as default.yaml will be overwritten with upgrades.

terse fiber
lyric violet
#

is custom.yaml used in its entirety and not merged like the yaml files would be for REST?

#

it would be good to have the changed templates in a different path that isn't touched by upgrades

terse fiber
#

custom.yaml is like default.yaml in that it only provides the mapping of collector-name to template. e.g.
Volume: volume.yaml

Yes, conf path helps keep your templates seperate from the out-of-the-box templates

lyric violet
#

right, but my question is custom.yaml would effectively replace the default.yaml if it is there in this case because of changes to the behavior in REST?

#

in the ONTAPI days, I could just put in my 'differences' into the custom.yaml and it would merge with the default.yaml

#

or maybe I'm confusing the REST yaml behavior from the top level behavior

#

is confused

terse fiber
#

It is confusing. There are collector templates and object templates. Collector templates are controlled by default.yaml, custom.yaml, and per-collector templates (not used as often). custom.yaml is merged with default.yaml for both zapi and rest.

ZAPI object templates are merged, but REST object templates are not. REST object templates replace

The examples in https://github.com/NetApp/harvest/issues/2330 may help too

#

Maybe this example helps. I created conf/rest/custom.yaml with these contents

When starting a poller with the Rest collector, in the logs I will see

which shows that all the default.yaml object templates are being loaded per normal and my Foo object template tries to load too, but fails because the path does not exist.

lyric violet
#

ok thanks, I'll add a custom.yaml for now and see what occurs

lyric violet
#

I have always added my templates as 'volume: volume_custom.yaml' in case of file collision (when upgrades occur) so moving them isn't really necessary, except for the collector templates

#

that way, I can track what changes occur when upgrades occur.

We also upgrade differently than just an rpm -Uhv....I do a rsync of the working configuration to a backup and then remove the old directory so that I get original files without conflict doing a fresh install.

#

we have a configuration mgmt tool that puts the files back to their needed state and we can simply restart it

#

I think I am safe in this way from upgrade overwrites