CLAMS App Metadata

Overview

Every CLAMS app must provide information about the app itself. We call this set of information App Metadata.

Format

A CLAMS App Metadata should be able to be serialized into a JSON string.

Input/Output type specification

Essentially, all CLAMS apps are designed to take one MMIF file as input and produce another MMIF file as output. In this section, we will discuss how to specify, in the App Metadata, the semantics of the input and output MMIF files, and how that information should be formatted in terms of the App Metadata syntax, concretely by using input and output lists and type vocabularies where @type are defined.

Note

CLAMS App Metadata is encoded in JSON format, but is not part of MMIF specification. Full json schema for app metadata is available in the below section. When an app is published to the CLAMS app directory, the app metadata will be rendered as a HTML page, with some additional information about submission. Visit the CLAMS app directory to see how the app metadata is rendered.

Annotation types in MMIF

As described in the MMIF documentation, MMIF files can contain annotations of various types. Currently, CLAMS team is using the following vocabularies with pre-defined annotation types:

Each annotation object type in the vocabularies has a unique URI that is used as the value of the @type field. However, more important part of the type definition in the context of CLAMS app development is the metadata and properties fields. These fields provide additional information about the annotation type. Semantically, there is no differences between a metadata field and a property field. The difference is in the intended use of the field. a metadata field is used to provide common information about a group of annotation objects of the same type, while a properties field is used to provide information about the individual annotation instance. In practice, metadata fields are placed in the view metadata (view[].metatadata.contains) and properties fields are placed in the annotation object itself. Because of this lack of distinction in the semantics, we will use the term “type property” to refer to both metadata and properties in the context of annotation type (I/O) specifications in the app metadata.

Type definitions in the vocabularies are intentionally kept minimal and underspecified. This is because the definitions are meant to be extended by an app developers. For example, the LAPPS vocabulary defines a type called Token, primarily to represent a token in a natural language text. However, the usage of the type can be extended to represent a sub-word token used in a machine learning model, or a minimal unit of a sign language video. If the app developer needs to add additional information to the type definition, they can do so by adding arbitrary properties to the type definition in action. In such a case, the app developer is expected to provide the explanation of the extended type in the app metadata. See below for the syntax of I/O specification in the app metadata.

Syntax for I/O specification in App Metadata

In the App Metadata, the input and output types are specified as lists of objects. Each object in the list should have the following fields:

  • @type: The URI of the annotation type. This field is required.

  • description: A human-readable description of the annotation type. This field is optional.

  • properties: A key-value pairs of type properties. This field is optional.

  • required: A boolean value indicating whether the annotation type is required as input. This field is optional and defaults to true. Not applicable for output types.

Simple case - using types as defined in the vocabularies

In the simplest case, where a developer merely re-uses an annotation type definition and pre-defined properties, an input specification can be as simple as the following:

{
  # other app metadata fields,
  "input":
  [
    {
      "@type": "http://vocab.lappsgrid.org/Token",
      "properties": {
        "posTagSet": "https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html"
      }
    }
  ],
  # and more app metadata fields,
}

In the above example, the developer is declaring the app is expecting Token annotation objects, with a posTagSet property of which value is the URL of the Penn Treebank POS tag set, verbatim, in the input MMIF, and all other existing annotation types in the input MMIF will be ignored during processing. There are some grammar of how this input list can be written.

  • The value of a property specification can be a verbatim string, or "*" to indicate that the property can have any value.

  • If the app expects multiple types of annotations, the input field should contain multiple objects in the list.

  • And if the app expects “one-of” specified types, one can specify the set of those types in a nested list. One nested list in the input specification means one required type.

  • And finally, if an input type is optional (i.e., required=false), it indicates that the app can use extra information from the optional annotations. In such a case, it is recommended to provide a description of differences in the output MMIF when the extra information is available.

For example, here is a more complex example of the simple case:

{
  # other app metadata fields,
  "input":
  [
    [
      { "@type": "https://mmif.clams.ai/vocabulary/AudioDocument/v1/" },
      { "@type": "https://mmif.clams.ai/vocabulary/VideoDocument/v1/" }
    ],
    {
      "@type": "https://mmif.clams.ai/vocabulary/TimeFrame/v5",
      "properties": {
        "label": "speech",
      }
      "required": false
    },
  ],
  # and more app metadata fields,
}

This app is a speech-to-text (automatic speech recognition) app that can take either an audio document or a video document and transcribe the speech in the document. The app can also take a TimeFrame annotation objects with label="speech" property. When speech time frames are available, app can perform transcription only on the speech segments, to save time and compute power.

Another example with even more complex input specification:

{
  # other app metadata fields,
  "input":
  [
    { "@type": "https://mmif.clams.ai/vocabulary/VideoDocument/v1/" },
    [
      {
        "@type": "https://mmif.clams.ai/vocabulary/TimeFrame/v5",
        "properties": {
          "timeUnit": "*"
          "label": "slate",
        }
      },
      {
        "@type": "https://mmif.clams.ai/vocabulary/TimeFrame/v5",
        "properties": {
          "timeUnit": "*"
          "label": "chyron",
        }
      }
    ]
  ],
  # and more app metadata fields,
}

This is a text recognition app that can take a video document and TimeFrame annotations that are labels as either slate or chyron, and have timeUnit properties. The value of the timeUnit property doesn’t matter, but the input time frames must have it.

Note

Unfortunately, currently there is no way to specify optional properties within the type definition.

Finally, let’s take a look at the output specification of a scene recognition CLAMS app:

{
  # other app metadata fields,
  "output":
  [
      {
        "@type": "https://mmif.clams.ai/vocabulary/TimePoint/v4/",
        "description": "An individual \"still frame\"-level image classification results.",
        "properties": {
            "timeUnit": "milliseconds",
            "labelset": ["slate", "chyron", "talking-heads-no-text"],
            "classification": "*",
            "label": "*"
        }
      }
  ],
  # and more app metadata fields,
}

Note that in the actual output MMIF, more properties can be stored in the TimePoint objects. The output specification in the app metadata is a subset of the properties to be produced that are useful for type checking in the downstream apps, as well as for human readers to understand the output.

Extended case - adding custom properties to the type definition

When the type definition is extended on the fly, developers are expected to provide the extended specification in the form of key-value pairs in the properties field. The grammar of the JSON object does not change, but developers are expected to provide a verbose description of the type extension in the description field.

Runtime parameter specification

CLAMS apps designed to be run as HTTP servers, preferably as stateless. When accepting HTTP requests, the app should take the request data payload (body) as the input MMIF, and any exposed configurations should be read from query strings in the URL.

That said, the only allowed data type for users to pass as parameter values at the request time is a string. Hence, the app developer is responsible for parsing the string values into the appropriate data types. (clams-python SDK provides some basic parsing functions, automatically called by the web framework wrapper.) At the app metadata level, developers can specify the expected parameter data types, among integer, number, string, boolean, map, and also can specify the default value of the parameter (when specified, default values should be properly typed, not as strings). Noticeably, there’s NO list in the available data types, and that is because a parameter can be specified as multivalued=True to accept multiple values as a list. For details of how SDK’s built-in parameter value parsing works, please refer to the App Metadata json scheme (in the below section).

Syntax for parameter specification in App Metadata

Metadata Schema

The schema for app metadata is as follows. (You can also download the schema in JSON Schema format from here.)

CLAMS AppMetadata

Data model that describes a CLAMS app.

Can be initialized by simply passing all required key-value pairs.

If you have a pre-generated metadata as an external file, you can read in the file as a dict and use it as keyword arguments for initialization. But be careful with keys of which values are automatically generated by the SDK.

Please refer to <CLAMS App Metadata> for the metadata specification.

type

object

properties

  • name

Name

A short name of the app.

type

string

  • description

Description

A longer description of the app (what it does, how to use, etc.).

type

string

  • app_version

App Version

(AUTO-GENERATED, DO NOT SET MANUALLY)

Version of the app.

When the metadata is generated using clams-python SDK, this field is automatically filled in

type

string

  • mmif_version

Mmif Version

(AUTO-GENERATED, DO NOT SET MANUALLY)

Version of MMIF specification the app.

When the metadata is generated using clams-python SDK, this field is automatically filled in.

type

string

  • analyzer_version

Analyzer Version

(optional) Version of an analyzer software, if the app is working as a wrapper for one.

type

string

  • app_license

App License

License information of the app.

type

string

  • analyzer_license

Analyzer License

(optional) License information of an analyzer software, if the app works as a wrapper for one.

type

string

  • identifier

Identifier

(partly AUTO-GENERATED)

IRI-formatted unique identifier for the app.

If the app is to be published to the CLAMS app-directory, the developer should give a single string value composed with valid URL characters (no /, no whitespace),

then when the metadata is generated using clams-python SDK, the app-directory URL is prepended and app_version value will be appended automatically.

For example, example-app -> http://apps.clams.ai/example-app/1.0.0

Otherwise, only the app_version value is used as suffix, so use an IRI form, but leave the version number out.

type

string

maxLength

65536

minLength

1

format

uri

  • url

Url

A public repository where the app’s source code (git-based) and/or documentation is available.

type

string

maxLength

65536

minLength

1

format

uri

  • input

Input

List of input types. Must have at least one element.

This list should iterate all input types in an exhaustive and meticulous manner, meaning it is recommended for developers to pay extra attention to input and output fields to include 1) all types are listed, 2) if types to have specific properties, include the properties.

This list should interpreted conjunctive (AND).

However, a nested list in this list means oneOf disjunctive (OR) specification.

For example, input = [TypeA (req=True), [TypeB, TypeC]] means``TypeA`` is required and either TypeB or TypeC is additionally required.

All input elements in the nested list must not be required=False, and only a single nesting level is allowed (e.g. input = [TypeA, [ [TypeB, TypeC], [TypeD, TypeE] ] ] is not allowed).

type

array

default

items

anyOf

#/definitions/Input

type

array

items

#/definitions/Input

  • output

Output

List of output types. Must have at least one element.This list should iterate all output types in an exhaustive and meticulous manner, meaning it is recommended for developers to pay extra attention to input and output fields to include

type

array

default

items

#/definitions/Output

  • parameters

Parameters

List of runtime parameters. Can be empty.

type

array

default

items

#/definitions/RuntimeParameter

  • dependencies

Dependencies

(optional) List of software dependencies of the app.

This list is completely optional, as in most cases such dependencies are specified in a separate file in the codebase of the app (for example, requirements.txt file for a Python app, or pom.xml file for a maven-based Java app).

List items must be strings, not any kind of structured data. Thus, it is recommended to include a package name and its version in the string value at the minimum (e.g., clams-python==1.2.3).

type

array

items

type

string

  • more

More

(optional) A string-to-string map that can be used to store any additional metadata of the app.

type

object

additionalProperties

type

string

additionalProperties

False

CLAMS Input Specification

Data model that describes input specification of a CLAMS app.

CLAMS apps are expected to have at least one input type, and each type must be defined by a @type URI string. If the type has specific properties and values required by the app, they can be described in the (optional) properties field. Finally, a human-readable verbose description can be provided in the (optional) description field for users.

Developers should take diligent care to include all input types and their properties in the app metadata.

type

object

properties

  • @type

@Type

The type of the object. Must be a IRI string.

type

string

maxLength

65536

minLength

1

format

uri

  • description

Description

A verbose, human-readable description of the type. This is intended to be used for documentation purpose for a particular use case of this annotation type and is not expected to be consumed by software. This description should work as a guideline for users to understand the output type, and also can be used as a expansion specification for the type definition beyond the base vocabulary.

type

string

  • properties

Properties

(optional) Specification for type properties, if any. "*" indicates any value.

type

object

default

additionalProperties

anyOf

type

integer

type

number

type

boolean

type

string

  • required

Required

(optional, True by default) Indicating whether this input type is mandatory or optional.

type

boolean

additionalProperties

False

CLAMS Output Specification

Data model that describes output specification of a CLAMS app.

CLAMS apps are expected to have at least one output type, and each type must be defined by a @type URI string. If the type has common properties and values generated by the app, they can be described in the (optional) properties field. Finally, a human-readable verbose description can be provided in the (optional) description field for users.

Developers should take diligent care to include all output types and their properties in the app metadata. To specify the property values, developers can use an actual value (for full match) or '*' (for any value).

type

object

properties

  • @type

@Type

The type of the object. Must be a IRI string.

type

string

maxLength

65536

minLength

1

format

uri

  • description

Description

A verbose, human-readable description of the type. This is intended to be used for documentation purpose for a particular use case of this annotation type and is not expected to be consumed by software. This description should work as a guideline for users to understand the output type, and also can be used as a expansion specification for the type definition beyond the base vocabulary.

type

string

  • properties

Properties

(optional) Specification for type properties, if any. "*" indicates any value.

type

object

default

additionalProperties

anyOf

type

integer

type

number

type

boolean

type

string

additionalProperties

False

CLAMS App Runtime Parameter

Defines a data model that describes a single runtime configuration of a CLAMS app. Usually, an app keeps a list of these configuration specifications in the parameters field. When initializing a RuntimeParameter object in python the value for the default field must be a string. For example, if you want to set a default value for a boolean parameter, use any of 'True', 'true', 't', or their falsy counterpart, instead of True or False

type

object

properties

  • name

Name

A short name of the parameter (works as a key).

type

string

  • description

Description

A longer description of the parameter (what it does, how to use, etc.).

type

string

  • type

Type

Type of the parameter value the app expects. Must be one of (‘integer’, ‘number’, ‘string’, ‘boolean’, ‘map’). When type is map, multivalued=true is forced, and when boolean, multivalued=false is forced.

Notes for developers:

When the type is map, the parameter value (still a single string from the users’ perspective) must be formatted as a KEY:VALUE pair, namely a colon-separated string. To pass multiple key-value pairs, users need to pass the parameter multiple times (remember type=map implies multivalued=true) with pairs in the colon-separated format.

Also, the VALUE part of the user input is always expected and handled as a string. If a developers wants to do more text processing on the passed value to accept more complex data types or structures (e.g., map from a string to a list of strings), it is up to the developer. However, any additional form requirements should be precisely described in the description field for users.

Finally, the same format is expected for the default value, if any. For example, if the default desired dictionary is {'key1': 'value1', 'key2': 'value2'}, the default value (used when initializing a parameter) should be ['key1:value1','key2:value2'] .

type

string

enum

integer, number, string, boolean, map

  • choices

Choices

(optional) List of string values that can be accepted.

type

array

items

anyOf

type

integer

type

number

type

boolean

type

string

  • default

Default

(optional) Default value for the parameter.

Notes for developers:

Setting a default value makes a parameter optional.

When multivalued=true, the default value should be a list of values.

When type=map, the default value should be a list of colon-separated strings.

anyOf

type

integer

type

number

type

boolean

type

string

type

array

items

anyOf

type

integer

type

number

type

boolean

type

string

  • multivalued

Multivalued

(optional, False by default) Set True if the parameter can have multiple values.

Note that, for parameters that allow multiple values, the SDK will pass a singleton list to _annotate() even when one value is passed via HTTP.

type

boolean

additionalProperties

False