Subject: [core] Status of SenML syntax

From: Christian Amsüss <[email-masked]>
Date: Wed, 5 Aug 2015 20:54:58 +0200

Hello Ari, hello everybody interested in SenML,

I've seen the currently pending questions w/rt SenML posed in the IETF93
slides[1], and the recordings[2] give me the impression that a list
[base dict, values, updating base dict, more values] is the way to go.

Would you accept patches to update the sections on JSON and CBOR
serializations and the examples? (If so, which is the master file to
patch, the markdown or the XML?)

Some points have been raised in the discussion I'd like to ask about /
comment on:

* Concerning "be consistent on being streamable" / order of items inside
  an event: Has there been further discussion on this afterwards? I do
  see the necessity in certain use cases (think {'sv':'...much
  text','n': oh that's where this all should have gone}), even though
  I'd figure they are rare; has a suggestion on how to deal with this
  been proposed?

  Unlike the base value case, this affects the XML representation as
  well.

* "It's usually not gigabytes": Just a side note -- my proxy for sleepy
  devices is using SenML for backup and replication purposes.
  Extrapolating, the largest SenML involved will hit the gigabyte in a
  few months.

* A google monitoring format has been mentioned that is similar to
  SenML; could I have some context on that?

Best regards
Christian

[1] https://www.ietf.org/proceedings/93/slides/slides-93-core-0.pdf
[2] http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_CORE_II&chapter=chapter_1 starting at 1:18

-- 
Christian Amsüss                      | Energy Harvesting Solutions GmbH
founder, system architect             | headquarter:
mailto:[email-masked]  | Arbeitergasse 15, A-4400 Steyr
tel:+43-664-97-90-6-39                | http://www.energyharvesting.at/
                                      | ATU68476614

---

From: Ari Keränen <[email-masked]>
Date: Wed, 19 Aug 2015 19:45:18 +0300

Hello Christian,

And sorry for the late reply; coming back from vacations and clearing my 
inbox now. See in-line.

> [base dict, values, updating base dict, more values] is the way to go.

At the meeting we seemed to have support for this, and no one objecting, 
but of course we're still looking forward for more feedback on the list.

> patch, the markdown or the XML?)

For sake of discussion (unless it's a typo or other very simple fix), I 
think it's better if you propose new text here on the list. Currently 
the XML at the SVN is the latest, but there was discussion that maybe we 
want to continue with github and/or markdown. No firm decisions yet on this.

>    I'd figure they are rare; has a suggestion on how to deal with this

There hasn't been discussion on this after the meeting except that 
couple of folks expressed their support for the approach and mentioned 
that they would go back to home organization and get feedback from their 
developers.

We didn't actually yet think about very long string values. Technically 
you're right that it could also cause similar issues, but in practice I 
think it would be a corner case of a corner case and hence I'd rather 
keep it as such. Would welcome more comments on this though.

>    Unlike the base value case, this affects the XML representation as

Maybe this is another reason for not doing that change.

>    Extrapolating, the largest SenML involved will hit the gigabyte in a

Interesting. I'd be inclined to say that perhaps SenML is not the right 
(or at least best) format then, but I'd be interested to hear more why 
you decided to use SenML for this.

>    SenML; could I have some context on that?

I'm not expert on that topic, so I'll leave this for other folks to answer.

Cheers,
Ari

---

From: Andrew Mcgregor <[email-masked]>
Date: Thu, 20 Aug 2015 10:26:11 +1000

>   SenML; could I have some context on that?

I mentioned that as it has some properties that are relevant, mostly that
it can be incrementally parsed and/or streamed.  The format itself is not
really interesting, and presently unpublished (basically just a key-value
list, with optional nested labelset-value dictionaries).

And you can probably guess that monitoring things at Google means
multi-gigabyte single data scrapes are reasonably common, which makes that
property important.  Keeping monitoring formats streamable reduces the
'datacenter tax'; turns out, a data center is a constrained environment in
some ways, not because you don't have the resources, but because you want
to minimise use of them for overhead activities to improve the efficiency
of the real work.