[tmcl-wg] Updates [was: Another TMCL use case]

[Robert Barta]

On Tue, May 20, 2003 at 11:14:11AM -0400, Dan Corwin wrote:
> * Dan Corwin
> 
> > > I can and do assume TMCL, in some independent process, will
> > > help confirm all those details of a TM someone has had the
> > > foresight and diligence to pre-declare within a schema file,
> > > and that such processes, by being used, will help to reduce
> > > errors in (some of) the XTM files I import or create.

> > Well, it depends what this "pre-declared schema" includes, so it may
> > be weak but it may also include all the necessary semantics.

> What I find *not* compelling is the batch-mode use case.  It has
> a place, but for my goals, it over-encapsulates your logic.

I think I understand what you mean. But only if the language adopts
the approach

  - there is one schema written in TMCL
  - there is one map loaded by a TM processor
  - a TMCL validator can mechanically decide whether the map is valid
    or not

does not necessarily mean this "batch-mode". No one can stop an
authoring environment to make use of a constraint like for example:

   forall $t [ * (car) ] =>
      exists $t [ oc (model) : * ]

Every time the user adds a car he is prompted to add an occurrence of
that given type.

And: as long as the schema is not checked the map can also be
inconsistent with it.

> That makes it of little help for bulk updates to a topic map, 
> as in this first additional TMCL use case am suggesting:

OK, TMCL is NOT a language for 'bulk updating'. Maybe TMQL or TM update
or whatever is, but not TMCL.

> > If I understand that correctly:
> > 
> >   - map is loaded
> >   - script information is extern or in the map?
> >   - interpreter walks over the map and fires the scripts?
> >   - scripts do something with the map?

> Correct.  Assume the scripts are outside of the map, and 
> change *only* the Strings in occurrences or the Roleplayers in
> associations - things expected when you populate a data base 
> table or update it from an import file.

OK, but this is not an application of TMCL either, it is a case for a
TM database which can process TMCL constraints at transaction points:

  open_transaction;
  foreach of my topics {
      fire the relevant script
      perform some operation
      TM may be 'inconsistent' now with TMCL schema
  }
  if (TM is valid) {
     commit;
  } else {
     rollback;
  }

The database will be able to translate the TMCL constraints into
operational code which can be checked at the end of an transaction.
Just like a relational database understands a specific set of
integrity constraints.

If you need more consistency then you may have to move the transaction
into the loop:

  foreach of my topics {
      open_transaction;
      fire the relevant script
      perform some operation
      TM may be 'inconsistent' now with TMCL schema
      if (TM is valid) {
          commit;
      } else {
        rollback;
      }
  }


> This use case would test, among other things, the speed of 
> TMCL validation.  Simple benchmarks help.
> 
> *  Jim Mason's Ferret system scans 2,000 words/second on a PC.
> *  I'd guess a simple update script is maybe 3-5 times faster.

Right.

Whether a 'off-the-shelf' software like such above database is fast
enough for you depends what you need. For many people a relational
database with OO programming like Java is completely sufficiently
... uhm ... fast, for others doing hi-speed image rendering like in
games this is not even a remote option.

> To avoid being obnoxiously slow, TMCL cannot be much slower.  That
> means it must perform at least a thousand tests/second.

Hmmm, if you need that speed (and you cannot adjust your approach or
architecture), then I would assume that you will have to write a
dedicated checker.

> I do not believe a complete TM could be validated that fast,
> especially in a remote batch process.

Again, there is nothing remote or running in a separate process.
You would maybe simply call a function tm.is_valid (schema) or let the
TM database do it.

> For speed reasons at least, TMCL must therefore check updates in a
> more linear-time fashion, using logic specific to each proposed
> change.

But only the application can know _what_ it will change and how the
change may or may not affect the validity of the map. There is NO
formal way to derive this automatically. The developer should know
whether a change endangers the validity or not.

> Here, I just want to establish Updates as an extra TMCL use case,
> distinct from the use case of global map validation.

I understand that, but this is the realm of a TM database because this
concept has only meaning together with transactions.

Again, the fact that a TMCL definition acts on a whole map is only a
mathematical abstraction to achieve generality. A particular
application may derive special rules in special circumstances like in
the following example:

-- TMCL constraint ---

  "all topics may only be of type cat or dog and nothing else"

-- application ----

  tm.add_topic (type => 'cat', name => 'catbert'); // no problem
  tm.add_topic (type => 'dog', name => 'dogbert'); // no prob
  tm.add_topic (type => 'entertainer', name => 'rhobert');  // raises exception in the TM database because it violates constraint

\rho