complex data transmission

edited from a section of a 1999 research note

by Paul Prueitt

The identification of useful patterns requires two essential ingredients. First, the real world must have a generator that produces an actual pattern that is repeated. This pattern can then be seen, sometimes, using measurements on co-occurrence of tokens in bit streams. The second ingredient is a specific knowledge of when the pattern begins and when it ends.

In simple cases, this is not an issue. For example the co-occurrence of terms in the distribution of word frequencies, or the co-occurrence of the range in which numerical data falls, is often within a context that easily establishes the beginning and end of the event.

However, most naturally patterns are complex, incomplete and / or not properly measured.

According to my 1999 researchnote, during complex transmission, (what I called) the Communication Manager (CM) was to provide a Fourier like spread of a signal into a specific decomposition involving the use of a substructural "vector’ basis.

A specific analogy to physical phenomenon provided me with the intuition. This analogy is based on my study of Karl Pribram’s vast literature in cognitive neuroscience and quantum neuroscience.

The vector basis, a mathematical notion from Fourier analysis, describes the nature of light by identifying energy wavelengths in the electromagnetic spectrum. The decomposition is also analogous to a bit stream to wave transformation seen in quantum mechanics. In data stream decomposition of signal, the set of repeated patterns in the signal is the signals’ "spectrum". This signal spectrum can describe the content of the stream. In my work since 1999 I have been able to specify that this decomposition is via a framework – such as the one developed by Sowa, Ballard or Adi.

The spread is followed by signal processing in a "spectral domain" and then by the inverse transformation of the signal into a new bit stream. This re-localization is called, "a collapse of the wave" and is where any "interpretation" of information must occur. "Knowledge" is regarded as only existing during this collapse. The CM follows this analogy in managing the complex transmission. The theory is grounded in neuropsychology and in the widely available experimental evidence regarding the processing of the flow of energy from the eye into brain regions.

In 1999 the intuition was strong but the technology had not been figured out. Also, the jargon I was using was an absolute barrier to most everyone else.

Figure 3.1: Simple and Complex transmission of data streams

In simple transmission, no processing of the data stream is allowed. The data transmission is said to be Newtonian and simple. In complex transmission, a "sign system" is created that allows the "cross level" decomposition of the meaning of specific information in specific contexts and having specific pragmatics. The sign system also provides structured annotation of context, and thus may shape the interpretation during the re-localization of information. If memory is available, in the form of a class of representations of substructural patterns, then the stratified communication theory proposed by Prueitt is realized.

Traversal of an information gap, generically called epistemic gaps in the complex systems literature, require either a forward transformation or an inverse transformation of the signal. It is assumed here that interpretation must involve the traversal of an epistemic gap. Once a data stream is decomposed into semantic invariance, various computational argumentations can occur in a spectral domain built from theme and / or concept spaces.

I then speculate on some things.

The semantic invariance may be statistically defined, as in the Dynamic Reasoning Engines (DREs) available from the company Autonomy Inc. The computational argumentation may be defined using quasi-axiomatic theory, Mill’s logic, and a class of procedures called "voting procedures".

The computational argumentation, in the substrate, changes the position of tokens in the theme or concept space. Recomposition uses voting procedures to perform the inverse transform and to produce a new data packet with well-established similarity and dissimilarity to the original data.

Ultimately, the natural objective of a knowledge extraction methodology is to produce a set of topics, perhaps organized into taxonomies. This set of topics is to be as complete as possible while respecting the content within areas that correspond to viewpoint. To respect the viewpoint, established by context, each area is to be treated separately. Thus, the measurement of consistency and completeness is made within areas and not across context.