Graffiti UI Config

Explore

Graffiti Tagging UI Config

What is a Tagging UI Config

Different tagging/annotation workflows serve different purposes. Users of our tagging tools need to access the correct set of tag definitions to fulfill their specific workflow. For instance, PMR users may need to view the tags in a specific order to fulfill a workflow. This order can be data-driven, which necessitates a standard way to represent the configuration needed to render these tags.

Goal

We have data driven UIs in almost all of our tagging applications albeit some gaps. Some of the gaps and potentials to improvement in how the data driven UI is implemented in the domain of editorial tagging are

Bespoke implementations: We have bespoke implementations that are effectively trying to do similar things, as seen in Megamind, Timebandits, Graffiti Timecode annotations UI config, and Graffiti Entity Tagging UI config. While this is largely due to the work being done at different points in time by different personnel, the team acknowledges that a unification and codification is necessary for simplicity.

Lack of Tracing: UI configs are persisted as large blobs of json objects. There is no traceability on when and who updated a specific section of the blob. This can potentially lead to erroneous config being present in the database and hence breaking the UI. Lack of tracebility would also mean lack of understanding on when a specific feature is added to the UI.

Lack of standardized naming conventions: As part of the multiple implementations we chose different naming conventions for defining different parts of the configuration. For example in megamind we have group/subgroup, in graffiti we have category/subcategory etc. Lack of standardization leads to confusion and potentially redoing work.

Potential for an uniform and extensible design: The UX for our bespoke tooling has been evolving and is subjected to evolve in the future as well. However we do have the potential to keep few things consistent for uniformity across timecode and entity tagging. In addition we also have the ability to keep the design open for future extensibility. For example, we need not restrict the max subcategories to 2 levels deep.

Potential for a fine grained access control in the UI config: We may have scenarios in the near future where a UI config for a particular workflow can be different for FTEs and Vendors. For example, vendors hired to do high subjectivity tagging needs to have only the high subjectivity part of the config tree to be rendered and rest of the tags to be hidden as part of the entity tagging. While this is achieved today in MFUI as a separate view, the implementation has tight coupling with the backend tag definition. We can also solve it by creating a new config tree just for vendor based high subjectivity tagging. However an alternate approach is view this as an access control problem and we can have hooks to hide/show relevant parts of the config tree based on the context like user info, workflow etc.

Unification of UI configs for Timecode annotations and Entity Tagging: Timecode annotations and entity tagging are very similar at heart - applying editorial metadata on either the entire entity or parts of the entity. However the existing data driven UIs have separate implementations. With Graffiti being the unified tool for both kinds of tagging there is a potential to standardize the UI config

The goal of this document is to build upon the existing strengths and define a standard way to represent a UI tag configuration, which can address most of the gaps mentioned above.

Tagging UI Config as a Tree

A tagging UI config is essentially a n-ary tree of nodes. Each node has a set of properties. There is a need to be flexible with the set of properties and they are subjected to modification whenever necessary. There are 2 types of nodes non-leaf nodes and leaf nodes. The leaf nodes are no different than the non leaf nodes except that the leaf nodes have a pointer to the underlying backend tag definition, which gives the taxonomy of the underlying tag like Storyline. Here is a sample class to capture the above representation.

The below psudeo code is not the db modeling but rather the high level representation of an object which gets hydrated to the UI.

class Node {

String nodeId;

String displayName;

List<Node> children;

TagDefinition underlyingTagDefinition;

int horizontalPosition;

int verticalPosition;

String parentNodeId;

String context/usecase;

boolean allowDescription;

boolean allowNotes;

boolean isTimecodeAnnotation;

TimecodeAnnotationType annotationType;

// any styling specific config? - OPEN QUESTION.

}

class TimecodeAnnotationType {

AUTOPLAY,

POSTPLAY,

SKIP_CREDITS,

EDITORIAL_MOMENT,

ADBREAK,

ADBREAK_METADATA

DGA_MARKER,

RECAP,

SKIP_CONTENT

}

Explanation of attributes

displayName

children - Pointer to the list of child nodes. Will be null for leaf nodes.

underlyingTagDefinition - can be named better but essentially the pointer to the underlying backend tag definition object which gives us the taxonomy of the tag.

verticalPosition(aka column position) - The relative position of a node with respect to its immediate parent’s children vertically. If we want to render all children belonging to a parent in new lines one below the other, the typical value of the verticalPosition is 0 with an increasing number for horizontalPosition. However if we want to render elements like impact, frequency which typically are side-side the values for those respective Node objects are 0 and 1 for example. This approach does not restrict a row to have only max of 2 children to be displayed side-side.

horizontalPosition(aka rowPosition) - The relative position of a node with respect to its immediate parent’s children horizontally. If we want to render all children belonging to a parent in a new line one below the other, the typical value of horizontalPosition is an increasing integer value like 0,1,2..n where there are n children for a parent.

parent - The pointer back to the immediate parent, can be used to navigate bottom up if necessary.

usecase - Some tags have double duty. They are applicable in different contexts. Ex: Wordbuilding in NPC vs Wordbuilding in Editorial Moments. While it can be argued that they need to have different tagIds with common taxonomy, we have a potential to support the current way of doing things through this attribute.

id - reference of the node

allowDescription - should the annotation contain the description field.

allowNotes - should the annotation contain the notes field.

isTimecodeAnnotation - whether the underlying hierarchy represents a ui config for a view corresponding to timecode annotations or not

timecodeAnnotationType - the concrete type of the annotation.

Validations in code could prevent saving configurations with invalid vertical and horizontal positions like making sure 2 child elements does not have the same horizontal and vertical positions etc.

Validations can also be performed to make sure if the ui config does not represent a timecode annotation then the timecodeAnnotationType should be null;

The above code snippet is not how the config is directly store in the database. The specifics of the database layout is given below.

class BackendTagValueNode {

String value;

String displayName;

boolean isTaggable;

List<TagValueNode> children;

String parent;

}

class BackendTagDefinitionNode {

List<TagValueNode> taxonomy;

Optional<String> tagId;

String tagName;

boolean allowMultiple;

TagDataType dataType;

}

enum TagDataType {

CHOICE, BOOLEAN, FREETEXT, INT

}

Aberrations

There are a few timecode annotation types without taxonomy. Ex: NPC - Trope and Other under Experimental Category. It is not a good practice in general to have tags without defined taxonomy and freeform inputs. The free form nature of tags makes the interpretation hard for downstream consumers and is subjected to human errors like spelling errors. There are a few possible solutions to support freeform tags in annotations and fit to this proposed ui config model.

TagDefinitionNode can have an empty/null tagId and an empty taxonomy. The mutation can always have a predefined way of setting the arbitary key-value pairs as we have a label and values attributes for descriptive timecode annotations.

Create a fake tagId with empty taxonomy and model these annotations as regular ones and not as descriptive annotations.

The better solution would be to push towards creating a first class tag

Future Proofing

Existing Use cases that will come to Graffiti

Adbreaks - Marker Annotation

IsPartnerPreferred as an additional metadata attribute.

Autoplay/Postplay - Marker Annotation

SkipCredits - Marker Annotation

DGAMarker - Marker Annotation

Other almost obsolete use cases like Point Of Interest etc.

Additional Metadata but they are not used.

Marker annotations are those that just capture the start/end timecodes. If we stick to the semantic of having a taxonomy backed way of applying tagging metadata at any grain(timecode or entity), then we can build a unified UI config that covers the span of all these use cases that Graffiti will eventually house. For example, we could have a boolean tag defined for capturing “IsPartnerPreferred” and model an adbreak as an annotation with backing tag.

Future use cases that could come to Graffiti

Capturing scene level metadata for ads congruence. While a lot of details about this use case are still yet to be decided, it is essentially similar to an “editorial moment” captured at a time frame. As long as we adhere to the convention of a tag based taxonomy, we can easily integrate these use cases both in terms of backend data model as well as the UI config necessary to paint these new cases on the page.

Data Modeling

DB layout

Possible DB schema modeling the uiconfignodes and their relationships with each other as well with the underlying tagdefinition.

⁠

Advantages?

Discoverability

Say we want to find all the workflows/views which have “Tag A” and replace it with “Tag B”, we can easily find those workflows and can modify only the relevant top nodes to make the necessary change. We can also bulk apply these changes and change all views at once without impacting the rest of the UI config.

Similarly, if we want to rename a top node which has the underlying nodes of Gore Impact and Gore Frequency to a new name, we can search for those top nodes and rename them without impacting the rest of the UI config.

Traceability

With the lastUpdatedBy and lastUpdatedAt columns we can always have a stamp of who made the latest changes and an integration with the activity logging system can provide us with fine-grained traceability at a node and view level.

Operation Ease

This design is in line with how we solved for the

rollup tagging⁠

where we noticed the problems with having huge blobs of json data being stored as-is. This model enables us to inject nodes as child nodes to any arbitrary nodes or create new root nodes easily.

Sample Workflows

Migrate Research Tagging from MFUI to Graffiti

Steps:

Define the workflow of research tagging - define what views it contains

Migrate Autoplay/Postplay & Adbreaks from Timebandits to Graffiti

TODO

Mint a new Tag

Steps

Users mint a new tag through SSTM - Involves creating a tagId, backend tag definition entry etc.

Graffiti immediately adds a new UI Config entry into the database representing

such⁠

an object, with a pointer to the underlying above created backend tag definition object.

Users mint new taxonomy for the above tag through SSTM

Newly added tags go to the rolluptagconfig table

Users can get to choose a view(new feature in SSTM) and add the newly added tag to the view at the place of their choice. This will result in creation of a new ui config node with appropriate attributes like position etc. We can always default to a standard display name per the tag definition and expose option to override it if necessary.

For example, they can choose Ad Creative Tagging View and pick the appropriate parent UI config node, say, Product and Brand(which will have a specific ui config node already existing in the db) and add this new config node as one of the child to it. Users can choose certain display attributes like “displayName”, “horizontalPosition” etc to specify where they want to put this new tag.

Note: This feature need not be exposed directly to users immediately but rather as an eng only feature to assess its usefulness.

Add an existing tag to an existing View

TODO

Hydrate the final UI Config

Call to fetch a view for a workflow

BE looks up the uiconfigworkflow and uiconfigview tables to fetch the root node.

The logic recursively looks into the records of uiconfignode and uiconfignode_relationships to build the entire tree.

The logic looks into the tagvalues_uiconfig and rolluptagconfig tables to fetch the tag definitions and filter down the taxonomy if certain tag values are not applicable

BE returns the hydrated tree.

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.