Notions such as Martin-Löf randomness () are attempts to give formal definitions that capture the idea of a particular infinite binary sequence being random. In the case of Martin-Löf randomness, for instance, a sequence is called random if it satisfies the countably many measure-one properties which are the complements of the effectively null sets in . One finds that Martin-Löf randomness is successful from a statistical point of view when it is noted that all such sequences satisfy important probabilistic laws such as the strong law of large numbers and the law of the iterated logarithm.

One fundamental result in algorithmic randomness is the characterization of the Martin-Löf random sequences in terms of the Kolmogorov complexity of their initial segments. Specifically, they are as complex as they can be:

Schnorr’s Theorem. if and only if there is a constant such that, for all ,

Below we will see a definition that picks out a subset of much as picks out a subset of , in the following sense: just as the sequences try their best to be truly random sequences, the functions in do their best to approximate random Brownian motion paths. We mentioned Schnorr’s theorem above for the sake of the parallel between that characterization of and the definition of the class .

Before going any further, let’s be clear about what we mean by Brownian motion:

Definition. Let be a probability space. A Brownian motion (or, Wiener process) parametrized by is a function — i.e. an assignment of a “path” to each — such that:

• almost surely.
• is almost surely continuous.
• For any , the random variables are independent, and have the respective normal distributions

Asarin and Pokrovskii first introduced the class as follows:

Definition. The set is defined to be the set of functions such that , and for which is linear with slope on each interval in . Note that every can be naturally coded as a binary string of length (based on whether it slopes up or down in the successive subintervals). Thus we can freely speak of the Kolmogorov complexity of such an .

Given that, we define the set of complex oscillations to consist of those for which there is a sequence such that:

• Each .
• The converge effectively to in the uniform norm: for every , .
• There is a constant such that, for all ,

The parallel between the last clause of this definition and the characterization of provided by Schnorr’s theorem is clear.

Fouché’s principal objective in the first paper is to establish a bijection between and . In particular, there is a computable isomorphism between and an encoding of . Each Martin-Löf random is assigned some complex oscillation , and given such a Fouché indicates a procedure to compute approximations to from the finite initial segments of . For the sake of this computability-theoretic analysis, Fouché wants explicit analytic representations of paths , which requires a particular approach to the concept of Brownian motion. That’s what I want to delve into next time.

I don’t know if these videos “should” be on YouTube, but they’re there for now, and there’s plenty of interesting material.

I’m also reasonably comfortable with basic ring theory and the manipulation of ideals and modules. I’d like to go up to Artin’s reciprocity theorem and at least be able to discuss it at the conceptual level. That means I should understand at least that much, and that will be the subject of my next post or two here.

I’m going to assume some level of comfort with rings and modules. In particular, the set of ideals has sums, products and the obvious -action

The story of class field theory begins when we ask what division could mean in the context of ideals. Everything takes place in a integral domain, where we can build a field of fractions for our ring, . A fractional ideal ought to be a submodule . We need more, though, because we also want to relate relate to via the multiplication, so we require that there is some for which . Note that this holds trivially in case is finitely generated.

It is easy to see that the multiplication of integral ideals extends to the fractions, and this gives us an associative operation with the unit . It is not the case that every fractional ideal has an inverse, but we do have a sort of “best approximation”

Then the product is trivially contained in . Moreover, if anything more were included that condition would fail, so if has an inverse in the semigroup, is it.

We also have the notion of a principle ideal , generated by a single element. This generalizes to fractional ideals as well, now allowing to range over the field . Clearly, every principle ideal is invertible, with inverse . We immediately have the notion of “principle fractional domain”, where all fractional ideals are principle. This would certainly imply that itself was a principle ideal domain, but the converse turns out to be true as well, since two principle ideals and are equal iff for some unit .

We finally get to the notion of class fields by taking the quotient of all fractional ideals by the subgroup principles. In general we only get back another monoid; if we want to get a class field group, we have to pass to the more restricted notion of a Dedekind domain. That will be the topic of my next post.

First of all, let’s consider an example of a classical measure-theoretic result. Suppose we have an integrable function . We call a point a Lebesgue point of provided that

Lebesgue Differentiation Theorem. Let be an integrable function. Then almost every point is a Lebesgue point of .

To say that “almost every” point is a Lebesgue point is to say that the set of Lebesgue points has measure . Recently, Pathak proved a version of the Lebesgue differentiation theorem in the spirit of algorithmic randomness. Her result follows a pattern that has been seen before, e.g. in V’yugin: (1) take some probabilistic or measure-theoretic result that holds almost everywhere, (2) add some computability-related hypothesis, (3) conclude that the result in fact holds for every Martin-Löf random point in the space.

OK, fine. So what is a Martin-Löf random point? To answer that, let’s consider a fuzzy moral question: what should count as a “random” element in our measure space? We might say that a random point shouldn’t be too special; so we might make this try:

Attempted Definition. A random point in a measure space is one that doesn’t satisfy any properties of measure , i.e. it is not contained in any null set.

This runs into the problem that, well, for any . Martin-Löf’s 1966 definition is based on the same general idea, but it gives an account of randomness that can be satisfied and turns out to have all sorts of interesting interactions with computability theory:

Definition. A random point is one that isn’t contained in any effectively null set.

Definition. An effectively null set is one of the form

where is a sequence of uniformly effectively open sets, for which for all .

Definition. A set is effectively open if it is a union of balls

with a computably enumerable subset of .

Basically, the sets narrow in on the null set in an effective manner. Any point in the space that cannot be pinned down in such an effectively null is what we call a Martin-Löf random point.

Alright, so returning to Pathak’s version of the Lebesgue differentiation theorem, what is her additional hypothesis? She restricts attention to functions that are not just integrable, but also:

Definition. A function is -computable if there is a computable sequence of polynomials such that

for all .

So, we only consider that can be effectively approximated by polynomials in the -norm. Pathak’s result is then:

Theorem. If is -computable, then every Martin-Löf random point is a Lebesgue point for .

I have put up some rough working notes in the Miscellania section of my web page that situate Pathak’s result in a conceptual framework developed by Mathieu Hoyrup and Cristóbal Rojas for working with algorithmic randomness in spaces other than Cantor space (where classical computability theory lives). Their work brings a unifying, systematic approach to results like Pathak’s and V’yugin’s (linked to above). I wrote the notes for my own benefit, as a way to get clear on some of the structure and details of the papers I’ve been talking about; perhaps someone else might find them helpful too.

Cover from Logicomix

Papadimitriou might be best known to logicians from his textbook with Harry Lewis on the theory of computation. But he already has an earlier foray into fiction. In 2003, his Turing: A Novel about Computation was published. Yes, it really is a novel, complete with a love triangle and digressions about, well, computation. (Note: the title refers to a character that is not the Alan you know.) I actually own a copy of that book; my mother the computer scientist gave it to me one birthday. At the risk of her seeing this, I only read the first little bit of that book. But that reflects more on my general relationship with novels than it does on the readability of that particular tome.

In any case, while I have never been one for comics or graphic novels, I’m sufficiently curious about Logicomix: An Epic Search for Truth to give it a shot. For one thing, all of the art by Alecos Papadatos and Annie Di Donna looks very nice. I won’t say anything about the contents of the book, having not had my hands on it. A preview is available at the Logicomix site, and Richard Zach gives his largely favorable take in the post linked to above.

My research has centered around implicature theory, a topic in the field of pragmatics.  This series will walk through the research I have done and the theory proposed in my thesis.  In doing so, it will start from the absolute basics; the series will be self-contained, presupposing only basic knowledge of logic and naive set theory.

This first post will serve to introduce pragmatics, the study of non-literal meaning.

What is pragmatics?

Pragmatics is the study of language in use. It can also be considered the study of non-literal meaning. Taken in the second sense, pragmatics carves the study of meaning in half with its sister science, semantics. Between characterizing literal and non-literal meaning, the science of linguistic interpretation is (ideally) fully spanned.

Pragmatics may best be understood as the complement of semantics. Semantics is the study of literal, cross-contextual meaning.

Characterizing semantics as the science of “meaning”, however, means nothing if we don’t properly understand the term “meaning”. Any theory of meaning must first pin the meaning of “meaning” if the enterprise is to get off of the ground.

Properly and fully analyzing the word “meaning” in a way that matches our pre-theoretic intuitions about the concept is a philosophical problem that still has no satisfactory solution. Any attempt to flesh out the concept either misses some important component of what we consider meaning or falsely attributes extra, incorrect properties to it. However, a working definition that has worked quite well for systematic scientific study is one promoted by philosopher Donald Davidson; if meaning is viewed as “truth-conditional meaning”, then familiar, rigorous methods in formal semantics (typically reserved for logical, constructed languages) can be used to study natural-language.

Viewing the meaning of a sentence as its truth-conditions (the conditions under which the sentence is true) fails to capture many aspects of meaning. It doesn’t capture shades of meaning, for instance, that separate one poetic statement from another. While other aspects of meaning are still important and worth explaining, there’s something to be said about the truth-conditional viewpoint. If a speaker knows the meaning of a particular statement, then it’s reasonable to say that the speaker knows when the statement is true or false. In other words, knowing the meaning of a sentence means knowing what makes the sentence true; if someone didn’t know what made a sentence true, then debatably the person doesn’t know what it means.

Thus, truth-conditional meaning is a proper subset of meaning at large. To explain and study meaning as a whole, one must explain and study the truth-conditions of sentences and how they are acquired. Though this is but a sub-part of the total problem of meaning, this sub-part is far from being adequately solved; even so, it is the most fruitful and quantified area of investigation into linguistic interpretation.

When we say things such as “Sally had a baby and got married”, semantics’ job is to tell us the truth-conditions of this sentence that are contained solely in the words used (and not in their connotations). Any semantic theory worth its salt will tell us that “Sally had a baby and got married” is true just in case Sally, in fact, had a baby and got married. Nothing shocking here; semantics, viewed in this way, seems like a trivial topic. Needless to say, as we consider general theories of interpretation that must account for the behavior of complicated logical and intensional operators, accurate semantic theorizing gets a lot harder.

Semantics cannot distinguish between the following two sentences:

A: “Sally had a baby and got married.”

B: “Sally got married and had a baby.”

Literally speaking, A and B both say the same thing. A is true just in case Sally, in fact, had a baby and got married. B is true in the exact same conditions, and so A and B have the same semantic content (written ||A|| = ||B||, where “||A||” is read “the interpretation of A”).

A good way to characterize pragmatics is to point out that, from the pragmatic point of view, A and B say quite different things. Sentence A suggests that Sally had a baby before she got married, while sentence B suggests things the other way around. The order of appearance for the conjuncts in these sentences matters; we tend to understand the order of appearance in a list of conjuncts as a temporal ordering, though nothing about the word “and” itself mandates this interpretation.

A pragmatic theory of the behavior of “and” should account for this non-literal difference between the meaning of A and B, whereas a semantic theory isn’t on the hook for such a thing.

Why care?

The difference between A and B above is so natural that it hardly seems to call for an explanation. However, there are plenty of reasons to care about characterizing such linguistic behavior mathematically.

The ease at which human beings incorporate non-literal speech in discourse is a fact worth explaining. Hardly anything humans utter contains purely literal meaning; what we utter doesn’t merely borrow from the words we utter, but also from general facts about human reasoning. In other words, context matters. Where, when, and how we say things play systematically into constructing the meaning of what we say.

Crafting an explanation of how language in use spans nearly every unique cognitive aspect of humankind. It means crafting a model of human reasoning as it applies to language and communication; this bridges what we find salient and what we expect others to find salient in the immediate context, shared knowledge, and other domains of belief. Explaining how we engage in linguistic reasoning involves explaining facts about reasoning at large.

For those with a more practical streak, pragmatics is an essential sub-problem in constructing artificial intelligence. Sophisticated artificial intelligence will require sophisticated communicative ability. Without being able to understand non-literal speech, our artificial agents will have serious trouble communicating reliably with humans. Such limitations necessarily limit their ability to perform; if artificial agents are to serve wider and more useful roles, their ability to converse naturally with laypeople will be absolutely essential.

Pragmatics then, broadly construed, is the attempt to predict natural, systematic linguistic inferences. Along with a good theory of semantics, a complete theory of pragmatics would afford us a truth-conditionally complete theory of meaning. Such a theory would be able to take any utterance and fully decode its meaning-in-context.

The problem in pragmatics that this series will be investigating is that of conversational implicature. The next post will be concerned with defining implicature and some of its sub-phenomena, one in particular to which my theory applies.

I’ve just put a draft of one such translation up on my web page. It is an excerpt from the 1894 edition of the Dirichlet-Dedekind Vorlesungen über Zahlentheorie. The selected portion focuses on the structure of the lattice of subfields of , and it spells out how Galois theory can be construed in the field-theoretic framework laid out by Dedekind (i.e. the modern take on Galois theory).

Any comments on the draft are welcome.

This describes Moebius transformations, which are maps of the complex plane which have the form

for fixed . This video really emphasizes how anything having to do with the complex numbers is really about rotation.

I don’t think we have a plug-in for embedded video yet. Maybe I’ll work on that later. Until then, enjoy the link. [UPDATE: We now have embedded youtube functionality, as can be seen. -- Ed]

Here are a few of the points which I think distinguish this approach:

Beneath the fold I will hint at a few of these points, and give some small detail for the first. Although the details are by no means worked out, I am certain that most of our methods for the real numbers (e.g., calculus, topological groups, measure theory) will carry over in a natural way. A significant exception may be  descriptive set theory (and other theories of the fine logical detail of ); in fact it seems to me that my notion of the continuum doesn’t even have a well-defined cardinality. Rather than a failing of this approach, I see this as a sign that set theory has injected unnecessary baggage into our theory of the number system.

These ideas are mostly idle musings, not in active development, and I’m releasing them to the wind. If anyone who happens by wants to develop these ideas, go ahead. You can take them for yourself (with an attribution) or I would be happy to work on a joint project. Otherwise, I’ll continue a haphazard development in these pages from time to time.

Let’s start by recalling the set-up for my model. The underlying machinery of the model will be the nonstandard theory of the natural numbers. A very nice introduction to these ideas can be found in the fourth chapter of Edward Nelson’s book Radically Elementary Probability Theory. Roughly speaking, a nonstandard model of has two nested models of the natural numbers. The raw ingredients are a pair of nested sets , an element and a bijective successor function which also acts as a successor . We call the elements of the standard elements; this is an important property, so we’ll give it a special name: . All the rest of the elements are non-standard. Corresponding to these two different types, we have  two notions of induction. “External” (true) induction says that the standard elements form a model for the natural numbers:

The internal induction axiom will apply to all of ; we want to say that and the compound propositions which include it are the only ones which are not inductible relative to the entire set. Thus for any proposition which does not involve

What all this means is that the standard numbers make up an initial segment of , and both sets act like the natural numbers. I like to think about the standard numbers as corresponding to the actions in our day-to-day life (like counting or dividing) or in our formal system (lengths of derivations), whereas the rest of consists of the huge numbers like astronomical distances or the number of atoms in an everyday object. At a practical level, if I can count to I can count to , but I’ll never be able to count the grains of sand on a beach even though there are a finite number of them. This non-inductible character to our actions is exactly the intuition which non-standard analysis captures.

Since and $0$ are inductible, the entire arithmetic/order theory of comes for free, and we can use all our usual intuitions about these operations. Just to emphasize this, we’ll start referring to the non-standard model with the bold notation: . Now we want to recover the theory of the continuum. To do this, we rescale until the gaps in the natural numbers are too small to see. Specifically, we fix a non-standard number , and we want the number to be our new unit. This doesn’t effect the addition or order operations at all, since these are indifferent to scaling. Multiplication, however, must be modified so that the new unit squares to itself. Using to distinguish this new operation, we would like to say

Of course, division by is not defined in , so really we should put a ceiling or floor operator on this definition.

But wait, the approximation is going to screw up our axioms like associativity and commutativity; sometimes we’ll end up off by one or two. To keep our operations well-behaved, we have to mod out by infinitesimal distances. Non-standard analysis to the rescue! We can define an equivalence relation by

The we can define the half-ray by . In fact, we’ll be focusing on the compact interval

It is a bit of work, which I will leave to another day, to see that addition, multiplication, order and (truncated) subtraction are well-defined on these equivalence classes. Division is a bit trickier, but I would argue that we don’t even really need it; it will be enough to work out division by two, and this is easily accomplished. Moreover, this squares with elevated position that is accorded to dyadic rationals in the theory.

Notice that the definition of involves the predicate. This means that we may need to be careful when using propositions defined from elements of . In some sense this is like taking limits, as it pushes us from a relatively simple domain (the internal theory of the natural numbers) into a more complicated situtation (the nonstandard theory). Because of this, we will usually try to make all our definitions and conduct our calculations before passing from to .

It is worth noting here that the structure I’ve just described cannot be the real numbers to which we are accustomed. To see this, consider the element . We assumed at the beginning that is a non-standard element, so and is not the zero element of . If we try to think about distances in , we must rescale so that . Then

Thus is a true infinitesimal. Moreover, there are lots of others, , all prearranged in a nice arithmetic hierarchy. Indeed, any real number defines a different scale of infinitesimal . These are precisely the “new entities” referred to above the fold. Although I haven’t worked though the details yet, I believe these infinitesimal scales ought to help us provide a more concrete picture of local properties, especially different degrees of differentiability.

I’m going to leave it at that for now. The next time I revisit this topic I’ll say some words about the function theory of my non-standard interval.

Sequential compactness theorem: Let be a countable language, and let be a sequence of -structures. Then there is a subsequence which elementarily converges to a limit -structure (with a countable universe).

The sequence elementarily converges to if, for any sentence of , implies that for sufficiently large . This notion and the stated result struck me as unfamiliar, but I figured maybe it was just me. However, model theorist John Goodrick also indicated that he’d never come across this notion of elementary convergence either.

Tao gives a quick proof of the theorem as a corollary to compactness, which goes roughly as follows:

Proof. The set of all theories in can of course be identified with where is the (countable) set of sentences from . Moreover, the space with the product topology is sequentially compact because it is a countable product of the sequentially compact . (See Proposition 9 here for instance.) So the sequence of theories in has a subsequence converging to some theory . Using the compactness theorem, it follows that is in fact a consistent theory, and we get a countable model . Without too much ado, one can verify that is indeed an elementary limit of the subsequence.

Note that the restriction on ’s countability is essential for this proof to go through, as an uncountable product of sequentially compact spaces need not be sequentially compact. Consider, say, . For let be the -th digit of ’s binary expansion, and let be the function . One can check that the sequence has no convergent subsequence.

One reason this formulation might not have gotten much play in model theory itself is that ultraproducts already give us elementary limits of sequences of structures, via Łoś’ theorem. Tao’s interest in elementary limits arises from his combinatorial pursuits. From his post:

The sequential compactness theorem also lets us construct infinitary limits of various sequences of finitary objects; for instance, one can construct infinite pseudo-finite fields as the elementary limits of sequences of finite fields. I recently discovered that other several [sic] correspondence principles between finitary and infinitary objects, such as the Furstenberg correspondence principle between sets of integers and dynamical systems, or the more recent correspondence principles concerning graph limits, can be viewed as special cases of the sequential compactness theorem;

Let’s consider an example of the kind of correspondence principle that Tao has in mind, one which was discussed in detail in an earlier post from Tao’s blog. First we have what is essentially a quantitative version of the statement that bounded, monotone sequences of real numbers converge:

Infinite Convergence Principle: For any and any , there exists an such that for all .

Tao goes through some machinations in order to finally arrive at a finitary version of the same principle, which he proves to be quickly interderivable with the infinitary version:

Finite Convergence Principle: If and , and with sufficiently large, then there is an such that for all .

As pointed out by Henry Towsner and Ulrich Kohlenbach in the comments on the post, the finite convergence principle is nothing more than the no-counterexample interpretation of the infinite convergence principle. (See this paper by Kohlenbach and Gaspar for more details.) There has been much recent work applying proof theory to the fields of analysis, combinatorics and ergodic theory. Kreisel’s no-counterexample interpretation comes up, for instance, in the formulation of a constructive mean ergodic theorem in the paper “Local stability of ergodic averages” by Avigad, Gerhardy and Towsner. Kohlenbach has a multitude of papers in this area, as well as the monograph Applied Proof Theory: Proof Interpretations and their Use in Mathematics.

We saw above that Tao indicated that the sequential compactness theorem subsumes things like this correspondence between the infinite and finite convergence principles (which is just an instance of the no-counterexample interpretation, or at its heart, Herbrand’s theorem) and correspondence principles concerning graph limits à la Elek and Szegedy (who use an ultraproduct construction). So it distills a common logical thread that runs through these results. I might write another post about the logical status of this sequential compactness theorem later. (Then again, I might very well not.)