Notes on architecture, by Mitchell Charity. Fall, 1994.
A big, picture document.
------------------------------------------------------------
* Contents
data model - some thoughts
board capture - Oct 20, data model bottom up
board capture - Oct 18, functional decompositino
What paper are we now quickly writing?
naming - Oct 18, board capture
overview doc - version mcharity Oct 06 15:53
data model (v.0)
Sep 21 high-level outline
------------------------------------------------------------
------------------------------------------------------------
------------------------------------------------------------
* data model - some thoughts
[converted stars to +'s]
From mcharity Thu Oct 20 15:28:00 EDT 1994
To: library2000-group@lcs.mit.edu
Subject: data model - some thoughts
Reply-to: mcharity@lcs.mit.edu
X-Phone: NE43-512:(617)253-6023 fax:258-8682 home:497-1506
I scribbled on the board a bit this morning.
Here are some thoughts on creating a data model.
The approach here is functional decomposition, and subsequent
recombination. I sketch the idea of decomposition, its encoding and
recombination, and parts of a decomposition for our data model.
+ Identify and describe modules of functionality (cluster the data)
[...]
+ Encoding and recombination
- A note on excessive flattening
There is the temptation to take multiple functional chunks and to
encode them with a single, flat, bag of name-value pairs. What is
going on here, and is it desirable? Consider two functional chunks,
each represented as a bag of name-value pairs. Merging them into a
single bag can be seen as inheritance, with its various associated
issues. There is no longer a simple way to refer to or reconstitute
the functional chunks. Names in the two bags may collide. Doing
something meaningful with collisions across functional boundaries is
difficult. Avoidance of collisions requires either similarly
difficult coordination, or creating new names which encode the
originating chunk. This latter is simply a somewhat crufty way of
_not_ merging them.
The conclusion? Probably insufficiently motivated above? Expect
extensive explicit nesting.
- A note on indirection
Conclusion - expect a general model of values being allowed to be
both real-values and value-references (identifiers like urns and urls,
and pairs of identifier,subpart-selection).
Argument - one wants the possibility of indirection whenever a value
(such as a name-value-pair bag addressing some function) appears more
than once, or when one wishes to permit multiple providers.
Efficiency is not a counter argument - one may include/cache a version
of the object along with the identifier.
- A note on keeping attributes well defined
Attribute lists can have the problem of the attributes being
attributes of different objects, sometimes subtly different. One
cause of this is excessive flattening.
For illustration, in a bibliographic record, the "author" may be an
attribute of the work, "edition" of a version of the work, and "pages"
of a particular printing. Trying to associate such a record with some
real or abstract document is trying.
This suggests that a well formed attribute list is a tuple of
(I was temped to call the first part the object-object, but resisted.)
Equivalently, written with a lis'thp:
(attribute-list (...))
- A note on associating attributes with objects
[Ah well, mind is gone. The rest of this doc is cursory.]
Attributes can be connect to objects externally, as above, or
"in-line", with an attribute explicitly identifying the object-object.
External has advantage of being able to refer to the attributes
explicitly and regarding multiple objects.
- A note on variable hiding (and explicit inheritance)
Ex: biblio for a doc, with a modifying a biblio for a version.
+ Some clusters
[The following is unannotated board capture, provided for whatever it
is worth.]
data description (things which might be also present in object)
data origin
data analysis (ex: page breaks in ascii ocr)
typing
linkage to doc description (page->image map)
doc description
real-page characteristic/space/spec
abstract -> real-page map
doc structure
doc structure -> page (abstract or real?)
versioning
biblio
name
series
things to do with biblio...
author authorities in biblio
mnc
------------------------------------------------------------
* board capture - Oct 20, data model bottom up
outline of a note on data model, bottom-up
clusters (of data based on function)
encoding
excessive flattening (inheritance)
indirection (bob, urn, or <- + modifier)
attrib-val-pair bag "(avpb refered-to-object (avps...))"
simple bag
attachment - inline vs taped together
variable hiding (ex: biblio for doc, then modifying biblio for version)
explicit inheretence
some clusters
data description (things which might be also present in object)
data origen
data analysis (ex: page breaks in ascii ocr)
typing
linkage to doc description (page->image map)
doc description
biblio
real-page characteristic/space/spec
abstract -> real-page map
doc structure
doc structure -> page (abstract or real?)
versioning
things to do with biblio...
author authorities in biblio
name
series
------------------------------------------------------------
* board capture - Oct 18, functional decomposition
classification of names:
nonresolvable - deadend - dont go further.
resolvable...
to resolve a name, need
recognize can use resolution mechanism
lookup mechanism
data maint, perhaps distributed
. bagging accessors
. accessors (1-1 access pathts = urls)
re "."...
identify and describe modules of functionality
branch
change spacial/temporal properties
doc meta data
doc rep meta data
biblio
hints
space translation
------------------------------------------------------------
* What paper are we now quickly writing?
From mcharity Tue Oct 18 21:58:59 EDT 1994
To: library2000-group@lcs.mit.edu
Subject: What paper are we now quickly writing?
Reply-to: mcharity@lcs.mit.edu
X-Phone: NE43-512:(617)253-6023 fax:258-8682 home:497-1506
Jeremy and I just talked for a few hours. Here are some quick notes.
The hour is late. I have tried for coherence. Apologies for lapses.
It seems a functional decomposition (identifying and describing needed
nuggets of functinality) is useful in understanding the design space
(the range of posible system designs). Two examples of nuggets are:
- "branching" function - giving a name and getting back a bunch of
names which have roughly equivalent meaning. For instance, trading a
urn for a url, when both identify the same object, and the url differs
only in being valid for a shorter time. (This was a bad example as it
illustrates a combination of "branching" and "namespace translation".
Better something which mapped from a urn to other urns, say ones for
commercial rapid delivery of the same stuff.)
- "document meta data" - sortof biblio info (though it looks like the
two should be separated out).
So what does this model do for us? It helps one say things like -
`The current URC spec is broken because it combines branching and doc
meta data in one seemless lump. The two have functions have different
constraints, and if/when they are both stored in a one text object, as
with URCs, they must be separable. Further, separating the various
needs makes it obvious that foo has been missed, and bar done
incorrectly.'
But thats not what I wanted to write about.
As we went to close down the conversation, it became clear that we
(atleast I) was unclear on just what the scope of "the paper" was.
A _brief_ discussion yielded the following...
There seem to be a variety of objectives in the air. One of them
seems to be to get a "paper" out to cs-tr by yesterday. But what
should it cover?
One breakdown of the range of pending papers is:
intro paper
big picture (ala inria)
the library-like subproblem ("research slice")
descrition
justification
our architecture decomposition
brief justification of notion
store
discovery
navigation (naming)
authorization (included as demand great, but handwaved away
with an encryption or capability (ie, its mainly
someone else's problem) argument)
illustrative example (perhaps should integrate with above)
including social context
the publisher+library+consumer model in particular
this is a `satisfies actual needs' issue
full architecture in detail paper
needs / design space analysis paper
subparts in great/research detail papers
replication, mumble, ...
So is the outline labled "intro paper " above an introductory paper,
or merely the introduction _to_ an introductory paper?
If we are putting together a done-by-yesterday intro paper, what is
the thrust? Posibilities include:
- a store/discover/nav decomposition is good and here's how a full
system might be built with them
- above, plus discussion of stuff like other services.
- above, plus input/output specs of modules. Forces resolution of
some significant choices which can be glossed over above.
- decomposition is good (why/how/implications). here's an example.
- needs analysis for a libraryish system. tradeoffs. example.
Have come out in rough order of difficulty.
QUESTION:
In so far as we are currently trying to get "a paper" written,
what/which paper is it?
mnc
------------------------------------------------------------
* naming - Oct 18 board capture
top view
names
tr lib data model
replication
different parts
other library stuff
naming ---
names
what
compatible vs incompatible spaces
syntax compatibility
syntax compatibility as test of `what is a name'
examples
urns urls handles authors
resolvers
atleast one
pro , con
atmost one
pro , con
to same/diff stuff
standard vs flexible typing
: 1 unbount
2 typed
3 std/fixed/limited typed
: self typing or no?
data
distribution
aliasing
comparison of distribution vs aliasing
objects
various - series, docs, versions, ...
biblio
other meta data
obj rep
------------------------------------------------------------
* overview doc - version mcharity Oct 06 15:53
From mcharity Thu Oct 6 15:58:31 EDT 1994
To: library2000-group@lcs.mit.edu
Subject: overview doc - version mcharity Oct 06 15:53
Reply-to: mcharity@lcs.mit.edu
X-Phone: NE43-512:(617)253-6023 fax:258-8682 home:497-1506
Should we be having another meeting to discuss the big picture?
It has been a few weeks now since the last one.
Mitchell
If people would find this more convenient as a web page than as email,
let me know...
----[version Oct 06 15:53]----
Our information infrastructure is improving, enabling the unbundling
of information services, and the specialization of their providers.
Specialization allows organizations to concentrate on addressing
specific narrow needs. An environment which supports specialization
allows competing implementations and evolutionary improvement. The
architectural challenge is to engineer a system which permits the
specialization of services, and their gracefull composition into large
evolving systems.
What infrastructure technology is changing, and how does it enable
specialization? Quality display, inexpensive computes, and improving
communication make it possible to deliver services to end user
equipment. Inexpensive disk, ram, and computes make it technically
inexpensive to offer information services. Inexpensive quality
communication means those services need not be centralized. <...>
But what are some examples of specialization, and how does one
determine where to cleave existing services into specialized parts?
Following are some examples, grouped by the type of cleavage, and
followed by a general discussion of cleavage selection.
One can specialize based on the kind of service. For example, one can
separate storage from discovery, storage from authorization, and
namespace maintenance from name resolution.
* Separating storage from discovery
* Separating storage from authorization
* Separating namespace maintenance from name resolution
One can specialize based on the "class of service" (Network-speak - is
there a more general way to say this?)
* Separation of storage services optimized for persistence,
availability, and speed.
One can vertically dis-integrate / decompose the creation of data.
* Separate out "authority" (in the library sense)
* Separate out quality control
* Separate out format conversion
* Separate out storage, distribution and awareness
linkage (commerce, data conversion, protocol diplomacy)
(a sideeffect of specialization)
----[]----
------------------------------------------------------------
* data model (v.0)
From mcharity Fri Oct 7 12:40:13 EDT 1994
To: library2000-group@lcs.mit.edu
Subject: data model (v.0)
Reply-to: mcharity@lcs.mit.edu
X-Phone: NE43-512:(617)253-6023 fax:258-8682 home:497-1506
Data, data, everywhere, and not a bit to flip.
- from C Rime of the Ancient Mutator
(or `a gremlin's frustrations with write-only persistent storage')
Well, its a bright and early on a sunny (hence the bright) Friday morn,
so it must be time for...
Gauging an OO data model (sorry)
This is an attempt to first explore the cognitive units in current
use, and then derive data types and functions. It is roughly
organized into the administrative context, the artifact, ...
* The Administrative Context
Series, and publishers, and reports, oh my.
- Series
What is a series? Here is a simple model, and a general one.
A simple model is an ordered set of sequentially numbered reports.
Something with reports named along the lines of SERIES-.
But as one looks around, exceptions accumulate.
MIT AI lab has a single number sequence which contains both AI Memos
and AI TRs. So there are AIM- and AI-TR-, but never
with the same number. These could be viewed, and in some
administrative respects are, as a single typed series. But officially
they are two series, sharing a sequence. Then again, this may change
at the whim of the publications office(/person). One series or two?
LCS has an unnumbered TR, "Barriers to Equality". It is not ordered
with respect to the usual LCS-TR-'s.
So a general model is a partially ordered set (bag?), of which one can
only, in general, ask whether a report is a member.
- Organizations and series
Organizations may have multiple series.
Series may have multiple organizations. The set (ordered) of
organizations may vary with each report, though often (usually?)
there are one or more primary organizations which are generally
present.
- Report
Consider a report called LCS-TR-278.
One day, a report labeled LCS-TR-278.b comes out. This suggest the
original was implicitly LCS-TR-278.a.
One day, you notice LCS-TR-278.b now has a different number of pages
and has been edited since you first got it.
So it looks like there are concepts
Report
Report-version (Acknowledged-version?)
Report-body (Version?)
- Reports and series
But wait, there is more.
A report may come out under multiple series. The series may share the
entire document, or have distinct covers and share only the body. The
series may be of grossly different organizations (w collaborations).
- Reports and bibliographic information
Bibliographic information includes a mix of acknowledged information
and artifact description. So while it often describes a version, the
body may drift out of sync, affecting fields like page count.
* The Artifact
So what about the Phys-Ob?
- PostScript
One has the PostScript file of a report. One has meta information
such as "this was generated using dvi2ps, so dont expect it to work
with ghostscript". One has meta information such as "this is a
faithful copy of the report sans the figure on page 5 which was cut
and pasted".
Sometimes you have two PostScript files, as when the report is long or
multiple generators were used. Some meta information is specific, and
some is shared.
Sometimes you have two PostScript files, and the are subtly different
versions of the same report. One color and one grayscale, with the
text ("the .red.gray. line") modified to match.
- A meta detour
Associated with documents is various structural meta information.
Pages are named ("i,ii,1,2,..."). There are gross divisions ("TOC,
Chapter1,Bibliography"), and fine divisions (footnotes, figures,
citations). All(?) are in users' concept spaces, and thus things they
may wish explicit.
One thus builds mappings between these conceptual structures and
document encodings.
PostScript, printed on paper pages, is perforce proper.
But as the media changes, things become less simple.
- Images
Images force several issues latent with PostScript.
[Brach. Got distracted by cruft. Will get back to this...]
- HTML
(- Scanned text)?
(- Attachments/multimedia)?
* Services
* Glue
* A straw model (with glue)
Anyone want to play volleyball?
Mitchell
------------------------------------------------------------
* Sep 21 high-level outline
From mcharity Wed Sep 21 12:16:27 EDT 1994
To: delphi@mit.edu, jeremy@the-tech.mit.edu, Saltzer@mit.edu, mcharity@lcs.mit.edu
Subject: high-level outline
Reply-to: mcharity@lcs.mit.edu
X-Phone: NE43-512:(617)253-6023 fax:258-8682 home:497-1506
Some brief thoughts...
display+communication+`powerful pcs' =>
`posible to deliver to end user equipment'
disk+ram => `very cheap to offer services'
communication => `those services need not be centralized'
decouplin/disentangling/decentralization =>
`architecture requires exposure/shadowing mechanisms'.
`shadowing' => `requires ability to dump - interation or id enumeration'
`efficient shadowing' => `requires full dump alternative - whats-new'
Decouple how? - Here is a division based on mumble mumble...
storage, index, client, namestuffs,
derivative services (third-party resell, proxys),
linkage (commerce, data conversion, protocol diplomacy)
------------------------------------------------------------
Return to Library 2000 home page.