Some computations allow shortcuts which can be used to speed them up. Consider the operation of raising a number to a positive integer power. It is possible, for example, to calculate by multiplying 13 by itself seven times,However, the shortcut of squaring three times considerably speeds up the computation,It is often quite difficult to determine whether a given computation can be sped up by means of such a trick. Computations that cannot be sped up are said to exhibit computational irreducibility.
While many computations admit shortcuts that allow them to be performed more rapidly, others cannot be sped up. Computations that cannot be sped up by means of any shortcut are called computationally irreducible. The principle of computational irreducibility says that the only way to determine the answer to a computationally irreducible question is to perform, or simulate, the computation. Some irreducible computations can be sped up by performing them on faster hardware, as the principle refers only to computation time.According to Wolfram (2002, p. 741), if the behavior of a system is obviously simple--and is say either repetitive or nested--then it will always be computationally reducible. But it follows from the principle of computational equivalence that in practically all other cases it will be computationally irreducible." Here, "practically all" refers to cases that arise naturally or from a simple..
A pairing function is a function that reversibly maps onto , where denotes nonnegative integers. Pairing functions arise naturally in the demonstration that the cardinalities of the rationals and the nonnegative integers are the same, i.e., , where is known as aleph-0, originally due to Georg Cantor. Pairing functions also arise in coding problems, where a vector of integer values is to be folded onto a single integer value reversibly.515202633414101419253236913182423581217112471112345Let(1)then Hopcroft and Ullman (1979, p. 169) define the pairing function(2)(3)illustrated in the table above, where . The inverse may computed from(4)(5)(6)(7)where is the floor function.51522303949604101623314050361117243241237121825331148131926002591420012345The Hopcroft-Ullman function can be reparameterized so that and are in rather than . This function is known as the Cantor function and is given by(8)illustrated in..
Universality is the property of being able to perform different tasks with the same underlying construction just by being programmed in a different way. Universal systems are effectively capable of emulating any other system. Digital computers are universal, but proving that idealized computational systems are universal can be extremely difficult and technical. Nonetheless, examples have been found in many systems, and any system that can be translated into another system known to be universal must itself be universal. Specific universal Turing machines, universal cellular automata (in both one and two dimensions), and universal cyclic tag systems are known, although the smallest universal example is known only in the case of elementary cellular automata (Wolfram 2002, Cook 2004).
A multiway system is a kind of substitution system in which multiple states are permitted at any stage. This accommodates rule systems in which there is more than one possible way to perform an update.A simple example is a string substitution system. For instance, take the rules and the initial condition . There are two choices for how to proceed. Applying the first rule yields the evolution , while applying the second rule yields the evolution . So at the first step, there is a single state (), at the second step there are two states , and at the third step there is a single state .A path through a multiway system arising from a choice of which substitutions to make is called an evolution. Typically, a multiway system will have a large number of possible evolutions. For example, consider strings of s and s with the rule . Then most strings will have more than one occurrence of the substring , and each occurrence leads down another path in the multiway system...
A Chaitin's constant, also called a Chaitin omega number, introduced by Chaitin (1975), is the halting probability of a universal prefix-free (self-delimiting) Turing machine. Every Chaitin constant is simultaneously computably enumerable (the limit of a computable, increasing, converging sequence of rationals), and algorithmically random (its binary expansion is an algorithmic random sequence), hence uncomputable (Chaitin 1975).A Chaitin's constant can therefore be defined as(1)which gives the probability that for any set of instructions, a particular prefix-free universal Turing machine will halt, where is the size in bits of program .The value of a Chaitin constant is highly machine-dependent. In some cases, it can even be proved that not a single bit can be computed (Solovay 2000).Chaitin constants are perhaps the most obvious specific example of uncomputable numbers. They are also known to be transcendental.Calude et..
If is a class of recursively enumerable sets, then the set of Gödel numbers of functions whose domains belong to is called its index set. If the index set of is a recursive set, then either is empty or contains all recursively enumerable sets.Rice's theorem is an important result for computer science because it sets up boundaries for research in that area. It basically states that only trivial properties of programs are algorithmically decidable.
Let denote the recursive function of variables with Gödel number , where (1) is normally omitted. Then if is a partial recursive function, there exists an integer such thatwhere is Church's lambda notation. This is the variant most commonly known as Kleene's recursion theorem.Another variant generalizes the first variant by parameterization, and is the strongest form of the recursion theorem. This form states that for each , there exists a recursive function of variables such that is a injection and if is a total function, then for all , ..., , and ,Yet another and weaker variant of the recursion theorem guarantees the existence of a recursive function that is a fixed point for a recursive functional.
A causal network is an acyclic digraph arising from an evolution of a substitution system, and representing its history. The illustration above shows a causal network corresponding to the rules (applied in a left-to-right scan) and initial condition (Wolfram 2002, p. 498, fig. a).The figure above shows the procedure for diagrammatically creating a causal network from a mobile automaton (Wolfram 2002, pp. 488-489).In an evolution of a multiway system, each substitution event is a vertex in a causal network. Two events which are related by causal dependence, meaning one occurs just before the other, have an edge between the corresponding vertices in the causal network. More precisely, the edge is a directed edge leading from the past event to the future event.Some causal networks are independent of the choice of evolution, and these are calledcausally invariant...
A set of integers is said to be one-one reducible to a set () if there is a one-one recursive function such that for every ,(1)and(2)Similarly, a set of integers is many-one reducible to set () if there is a recursive function such that for every ,(3)and(4)Here, the two reducibility relations are reflexivity and transitivity.One-one reducibility implies many-one reducibility. Any set reducible to a recursive set is recursive itself. Any set reducible to a recursively enumerable set is recursively enumerable itself. The above facts are commonly used in recursive undecidability proofs done by reduction to nonrecursive sets.Two sets are one-one/many-one equivalent if each of them is one-one/many-one reducible to the other. One-one equivalence, many-one equivalence, and recursive isomorphism are all equivalence relations.If sets and are recursively isomorphic, then they are one-one equivalent and vice versa.If a class (also called..
A multiway system that generates causal networks which are all isomorphic as acyclic digraphs is said to exhibit causal invariance, and the causal network itself is also said to be causally invariant. Essentially, causal invariance means that no matter which evolution is chosen for a system, the history is the same in the sense that the same events occur and they have the same causal relationships. The figures above illustrate two nontrivial substitution systems that exhibit the same causal networks independent of the order in which the rules are applied (Wolfram 2002, pp. 500-501).Whenever two rule hypotheses overlap in an evolution, the corresponding system is not causally invariant. Hence, the simplest way to search for causal invariance is to use rules whose hypotheses can never overlap except trivially. An overlap can involve one or two strings. For example, does not have any overlaps. However, can overlap as , and the set of strings..
Determination of whether predicate is true or false for any given values of , ..., is called its decision problem. The decision problem for predicate is called recursively decidable if there is a total recursive function such that(1)Given the equivalence of computability and recursiveness, this definition may be restated with reference to computable functions instead of recursive functions.The halting problem was one of the first to be shown recursively undecidable. The formulation of recursive undecidability of the halting problem and many other recursively undecidable problems is based on Gödel numbers. For instance, the problem of deciding for any given whether the Turing machine whose Gödel number is is total is recursively undecidable. Hilbert's tenth problem is perhaps the most famous recursively undecidable problem.Most proofs of recursive undecidability use reduction. They show that recursive decidability..
Turing machines are defined by sets of rules that operate on four parameters: (state, tape cell color, operation, state). Let the states and tape cell colors be numbered and represented by quadruples of ordinal numbers. Then there exist algorithmic procedures that sequentially list all consistent sets of Turing machine rules. A set of rules is called consistent if any two quadruples differ in the first or second element out of the four. Any such procedure gives both an algorithm for going from any integer to its corresponding Turing machine and an algorithm for getting the index of any consistent set of Turing machine rules.Assume that one such procedure is selected. If Turing machine is defined by the set of quadruples whose index is , then is called the Gödel number of . The result of application of Turing machine with Godel number to is usually denoted .Given the equivalence of computability and recursiveness, it is common to use Gödel..
A set of integers is said to be recursively enumerable if it constitutes the range of a recursive function, i.e., if there exists a recursive function that can eventually generate any element in (Wolfram 2002, p. 1138). Any recursive set is also recursively enumerable.The union and intersection of two recursively enumerable sets are also recursively enumerable.Recursively undecidable problems give examples of recursively enumerable sets that are not recursive. For example, convergence of is known to be recursively undecidable, where denotes the Turing machine with Gödel number . Hence the set of all for which is convergent is not recursive. However, this set is recursively enumerable because it is the range of defined as follows:(1)A set is recursive iff both and its complement are recursively enumerable. This provides an approach to constructing additional sets that are not recursively enumerable. In particular, the..
A set of integers is said to be recursive if there is a total recursive function such that for and for . Any recursive set is also recursively enumerable.Finite sets, sets with finite complements, the odd numbers, and the prime numbers are all examples of recursive sets. The union and intersection of two recursive sets are themselves recursive, as is the complement of a recursive set.
The term "recursive function" is often used informally to describe any function that is defined with recursion. There are several formal counterparts to this informal definition, many of which only differ in trivial respects.Kleene (1952) defines a "partial recursive function" of nonnegative integers to be any function that is defined by a noncontradictory system of equations whose left and right sides are composed from (1) function symbols (for example, , , , etc.), (2) variables for nonnegative integers (for example, , , , etc.), (3) the constant 0, and (4) the successor function .For example,(1)(2)(3)(4)defines to be the function that computes the product of and .Note that the equations might not uniquely determine the value of for every possible input, and in that sense the definition is "partial." If the system of equations determines the value of f for every input, then the definition is said to be "total."..
For any constructible function , there exists a function such that for all functions , the following two statements are equivalent: 1. There exists an algorithm such that for all inputs , computes in volume . 2. is constructible and . Here, the volume is the combined number of active edges during all steps, which is the number of state-changes needed to run a certain Turing machine on a particular input.
A Turing machine is called deterministic if there is always at most one instruction associated with a given present internal state/tape state pair . Otherwise, it is called nondeterministic (Itô 1987, p. 137).In prediction theory, let be a weakly stationary process, and let be a subspace spanned by the (with ). If is independent of so that for every , then is said to be deterministic (Itô 1987, p. 1463).
A set of integers is productive if there exists a partial recursive function such that, for any , the following holds: If the domain of is a subset of , then is convergent, belongs to , and does not belong to the domain of , where denotes a recursive function whose Gödel number is .Productive sets are not recursively enumerable.
A recursively enumerable set is creative if its complement is productive. Creative sets are not recursive. The property of creativeness coincides with completeness. Namely, set is creative iff if it is many-one complete.Elementary arithmetic formulas are built up from 0, 1, 2, ..., , , , variables, connectives, and quantifiers. The set of all true arithmetic formulas is productive. Informally speaking, this means that no axiomatization of arithmetic can capture all true formulas and nothing else. For example, consider Peano arithmetic. Under the assumption that no false arithmetic formulas are provable in this theory, provable Peano arithmetic formulas form a creative set.
A function is said to be constructible if some algorithm computes it, in binary, within volume , i.e., . Here, the volume is the combined number of active edges during all steps, which is the number of state-changes needed to run a certain Turing machine on a particular input.
As first shown by Meyer and Ritchie (1967), do-loops (which have a fixed iteration limit) are a special case of while-loops. A function that can be implemented using only do-loops is called primitive recursive. (In contrast, a computable function can be coded using a combination of for- and while-loops, or while-loops only.) Examples of primitive recursive functions include power, greatest common divisor, and (the function giving the th prime).The Ackermann function is the simplest example of a well-defined total function that is computable but not primitive recursive, providing a counterexample to the belief in the early 1900s that every computable function was also primitive recursive (Dötzel 1991; Wolfram 2002, p. 907).
In machine learning theory and artificial intelligence, a concept over a domain is a Boolean function . A collection of concepts is called a concept class.In context-specific applications, concepts are usually thought to assign either a "positive" or "negative" outcome (corresponding to range values of 1 or 0, respectively) to each element of the domain . In that way, concepts are the fundamental component of learning theory.
Inspired by computer simulations of fossilized worms trails published by Raup and Seilacher (1969), computer scientist Mike Paterson at the University of Warwick and mathematician J. H. Conway created in early 1971 a simple set to rules to study idealized worms traveling along regular grids. Mike Beeler of the MIT Artificial Intelligence Laboratory subsequently published a study Paterson's worms in which he considered paths on a triangular grid (Beeler 1973).The following table summarizes the number of steps required for a number of long-running worms to terminate (Rokicki).patternsteps to terminate10420151042020104202212521211420221infinite14202241450221infinite14502241525115201414221451422454142Paterson's worms are featured in the 2003 Stephen Low IMAX film Volcanoesof the Deep Sea...
An abstract machine is a model of a computer system (considered either as hardware or software) constructed to allow a detailed and precise analysis of how the computer system works. Such a model usually consists of input, output, and operations that can be preformed (the operation set), and so can be thought of as a processor. Turing machines are the best known abstract machines, but there exist many other machines as well such as cellular automata.Abstract machines that model software are usually thought of as having very high-level operations. For example, an abstract machine that models a banking system can have operations like "deposit," "withdraw," "transfer," etc.An abstract machine implemented in software is termed a virtualmachine, and one implemented in hardware is called simply a "machine."..
A universal cellular automaton is a cellular automaton which, like a Turing machine, exhibits universality. von Neumann proved that an automaton consisting of cells with four orthogonal neighbors and 29 possible states would be capable of simulating a Turing machine for some configuration of about cells (Gardner 1983, p. 227).The outlines of a proof that the two-dimensional game of life outer-totalistic cellular automaton is universal were given by Berlekamp, Conway, and Guy (1982) and independently by Gosper (Gardner 1983, pp. 250-253). Around 2000, a Turing machine was explicitly implemented in life by P. Rendell (Rendell, Adamatzky 2001). While Rendell's machine can be made into a "true" universal computer simply by making his tape infinite, he neither noted this fact nor provided an actual construction of a universal Turing machine. Subsequently, on November 11, 2002, P. Chapman constructed..
The mathematical study of abstract computing machines (especially Turingmachines) and the analysis of algorithms used by such machines.A connection between automata theory and number theory was provided by Christol et al. (1980), who showed that a sequence is generated by a -automaton iff the formal power series with coefficients is algebraic on the field of rational elements , where and are polynomials with coefficients in the finite field .
A -automatic set is a set of integers whose base- representations form a regular language, i.e., a language accepted by a finite automaton or state machine. If bases and are incompatible (do not have a common power) and if an -automatic set and -automatic set are both of density 0 over the integers, then it is believed that is finite. However, this problem has not been settled.Some automatic sets, such as the 2-automatic consisting of numbers whose binary representations contain at most two 1s: 1, 2, 3, 4, 5, 6, 8, 9, 10, 12, 16, 17, 18, ... (OEIS A048645) have a simple arithmetic expression. However, this is not the case for general -automatic sets.
A Turing machine which, by appropriate programming using a finite length of input tape, can act as any Turing machine whatsoever. In his seminal paper, Turing himself gave the first construction for a universal Turing machine (Turing 1937, 1938). Shannon (1956) showed that two colors were sufficient, so long as enough states were used. Minsky (1962) discovered a 7-state 4-color universal Turing machine, illustrated above (Wolfram 2002, p. 706). Note that the 20th rule specifies that the Turing machine should halt, as indicated by leaving the head stationary and not changing its state. Upon conversion to a 2-color machine, Minsky's universal Turing machine requires 43 states.Comparatively little more was published about small universal Turing machines until Rogozhin (1996) found examples with the numbers of states and colors given by (24, 2), (10, 3), (7, 4), (5, 5), (4, 6), (3, 10), and (2, 18) (Wolfram 2002, p. 1119).A 2-state..
A Turing machine is a theoretical computing machine invented by Alan Turing (1937) to serve as an idealized model for mathematical calculation. A Turing machine consists of a line of cells known as a "tape" that can be moved back and forth, an active element known as the "head" that possesses a property known as "state" and that can change the property known as "color" of the active cell underneath it, and a set of instructions for how the head should modify the active cell and move the tape (Wolfram 2002, pp. 78-81). At each step, the machine may modify the color of the active cell and change the state of the head. After this, it moves the tape one unit to the left or right.Turing machines are implemented in the WolframLanguage as TuringMachine.A generalization of the Turing machine in which the head is allowed to move steps in either direction was considered by Macura (2005).A template for specifying..
An encoding is a way of representing a number or expression in terms of another (usually simpler) one. However, multiple expressions can also be encoded as a single expression, as in, for example,which encodes and uniquely as a single number.000011102023114205More generally, any list of positive integers can be uniquely encoded using a Gödel number (Wolfram 2002, p. 1120).
Let be a set and a collection of subsets of . A subset is shattered by if each subset of can be expressed as the intersection of with a subset in . Symbolically, then, is shattered by if for all , there exists some for which .If is shattered by , one says that shatters .There are a number of equivalent ways to define shattering. One can easily verify that the above definition is equivalent to saying that shatters ifwhere denotes the power set of . Yet another way to express this concept is to say that a set of cardinality is shattered by a set if where here,In the field of machine learning theory, one usually considers the set to be a sample of outcomes drawn according to a distribution with the set representing a collection of "known" concepts or laws. In this context, saying that is shattered by intuitively means that all of the constituent outcomes in can be known by knowing only the laws in ...