[Baroni, 2020] Ideas for a useful deep-agent protolanguage -- Baroni -- 2020

⁠

Baroni, Marco. "Rat big, cat eaten! Ideas for a useful deep-agent protolanguage." arXiv preprint arXiv:2003.11922 (2020).⁠

⁠

Coming back to my application scenario ... a simplified laboratory setup could involve multiple agents that have to jointly maintain a set of household goods in stock.

* Minimally, there are two agents.

* A pantry agent keeps track of the current supply level, that is constantly in flux as some items are consumed, others go bad, etc..

* A shopping agent would have to provide missing goods, possibly with budgetary restrictions.

This scenario should meet my desiderata:

* realistic perceptual input,

* new objects steadily popping up,

* at least some degree of predication needed (“tomatoes green”, “potatoes out”, “mozzarella rotten”).

Main Concern: how should a useful protolanguage for deep agents look like?

to develop language-like communication skills, we cannot just train them to reproduce statistical regularities in static linguistic corpora

a property is asserted of an object, a predication ... a feature that natural languages would likely denote through an adjective, but it could also be an action or state typically denoted by a verb.

parataxis defines the tendency to juxtapose simple clauses, instead of sub- ordinating them to construct complex sentences. Cf. “I came, I saw, I conquered” vs. “After coming, having seen, I decided to conquer”

I conjecture that agents can be very useful without the need to think in abstract terms ... We can moreover let the language be quite rough in its ways of reference, akin to the children’s two-word stage.

Rationale

Both on the phylogenetic and on the ontogenetic scale, human language does not appear all at once in fully-formed garb

once machines are endowed with a minimally sensible protolanguage, I expect humans would be willing and able to make an effort to understand what they mean

Decades of failed attempts in ML/AI suggest that manual language coding is invariably a bad idea, as shown by the current dominance of end-to-end deep networks over systems relying on explicit linguistic structures in virtually all domains of natural language processing

Desired characteristics

We need the emergent language to have the following characteristics

use words to categorize inputs lying on a continuous space into distinct classes

seamlessly create new words when new concepts are encountered

express combinations of arguments and predicates

Categories and new words

Languages strike a good balance between splitting the world into classes that are so detailed as to be of no use (my keyboard at 3.30pm on February 14th 2020) and classes that are so broad as to be again useless (“entity”).

Word coinage Agents should be able to refer to new things that might appear on their horizon, such as a new type of cereal (or new ways to categorize existing things, like when you learned that a subset of the grains you already knew were cereals).

Argument-predicate combinations

Agents able to communicate about an open set of primitive categories by adding new words to their repertoire would already be quite a feat.

The expressiveness of the language would however enormously increase if object-denoting and action-state-or-property-denoting words could be combined in simple predication structures.

Paratactic concatenation

Refer to more complex situations by paratactic concatenation of unary predications (“rat eating . . . cat eaten”). I think however that we already have enough on our plate dealing with the single predication case.

Aspects better left for later

the protolanguage need not support the further expressive power complex verbal morphology and sentential scaffoldings would provide: for example, the ability to express counterfactuals, hypotheticals, belief degrees, etc.

we probably do not need to worry about Genericity. Does “rat big” mean that rats are big in general, or that there is a big rat in the kitchen?

Speech acts. In (2a), are you asking me if the rat is big, or informing me that it is?

Expressing more complex concepts through massive parataxis. The agents can concatenate multiple predications to articulate more complex concepts, leaving the appropriate binding to context.

So, “a rat is eating the cat” could be expressed by something like: rat eating. . . cat eaten

Similarly, temporal or causal links could be expressed by parataxis aided by iconicity (that is, ordering predications in the way in which the corresponding events occurred, placing the cause before the effect, etc.): (REF 5) a. rat running. . . cheese reached. . . rat stopping b. rat hungry... rat eating

Coreference can be left implicit or provided by sheer repetition of the same nouns, as in the examples in (REF 5)

Personal pronouns, proper nouns, quantification. A lot of things that formal semanticists have very rightly argued cannot be treated as standard nouns or adjectives could be treated as standard nouns or adjectives in our protolanguage

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.