Crumbles

Measuring life

2009-11-10T20:07:00.000-08:00

Life is measured not in years but in a number of dreams of youth you have let go.

Thesis of darkness

2009-10-01T22:52:00.000-07:00

In an evolving system only around one twelfth of substance contributes to the observable cause and consequence relations, the rest is darkness.

Dark matter and dark energy in physics.
Dark DNA in biology – junk or non-coding DNA.
Dark money in economics – money made by promises, in opposite to visible coin, plastic, and digital money.
Dark information – unexplainable sequences of events.

Wealth and Income

2009-09-29T20:44:00.000-07:00

An interesting observation. I and many of my friends have to work to pay our debts. We are intelligent proletarians. A proletarian is a person who possessing neither capital nor production means, must earn his living by selling his labour. A non-proletarian may work, but he does not have to in order to support his living. We are paying out mainly our mortgages, which are mainly the loans for the houses we live in. And most of the value of the houses is the land. So basically we work hard to pay out the land we live on. That makes me ponder.

A simple model. A number of people are willing to buy a resource which is not producible by labour, for example, land. The market price for the land rises up to the level of the limit of competitive abilities of those people. If the people are desperate they borrow from the bank some kind of paper. By doing this they sell their future labour. The owner puts this paper back to the bank, and the bank cancels the paper. When future comes, people have to work to pay back the borrowed paper producing a lot of consumable products. If the amount of products is more than enough, they have to produce all kinds of luxuries to please the owners of the bank and the resource previous owners. When the demand for the resource declines, the prices fall, leaving years of hard labour spent on something, which those people not really required at the first place.

I always suspected that the slavery era has not ended. It transformed. People, who have to work and earn for living, pay taxes. They pay taxes directly through their income and indirectly through businesses they work for, and through the sale taxes. But some who do not have to work usually pay much less taxes, because their lives are businesses with tax deductible expenses. Am I a supporter of this kind of system? Would a simple solution as taxing wealth (not wealth taxing) instead of personal income make all people happier? That makes me ponder.

Bomb Problem (part II)

2009-04-13T19:29:00.000-07:00

Assume that point A is at location -1 and point B is at 1. If one source produced both A and B and the next shot is expected to be produced by the same source, then the best fit is the normal distribution with mean 0 and dispersion 1: N(0,1,x) - gives the maximum probabilities for points A and B. On the other hand, if different sources produced A and B, the best guess would be that the third shot is normally distributed with one of the sources F(x). Now since no information present about whether one or two sources produced A and B, give the same chances to either way:

f(x)=(N(0,1,x)+F(x))/2

For F(x), give the same chances to points A and B:

F(x)=(F(-1,x)+F(1,x))/2

And the best guess for F(y,x) would be N(y,(x-y)/2,x), because the only information about the dispersion for F(y,x) is the distance between x and y, and the dispersion (x-y)/2 is the best fit. Note, that the function F(y,x) is not proper distribution for variable x, because the probability is given for fixed dispersion and it depends on x, i.e. to be normalized the dispersion has to be fixed. The overall distribution then is:

f(x)=(N(0,1,x)+(N(-1,(x+1)/2,x)+N(1,(x-1)/2,x))/2)/2

where

N(y,s,x)=1/sqrt(2Pi*s²)*Exp(-0.5(x-y)²/s²)

The graph for function f(x) is presented above. The minimum values are at points: -0.47 and +0.47. This means that the safest place is about a quarter distance from either end.

Kernel Density Estimation: Multikernel and Bomb problem

2009-04-02T21:27:00.000-07:00

Given a sequence of independent identically distributed random variables X₁, X₂, ..., X_N with common probability density function f(x), how can one estimate f(x)? (Old good KDE problem)

Let us consider the following simple question. You are on the road and you cannot go off side the road. Two bombs fall at points A and B in no particular order (for example you can see only two shell holes, but do not know which one was the first). Where is the safest place on the road if you expect the third bomb to fall? Obviously it is as far as possible from the both points. But what about the safest place between A and B? Is it the middle? Is it the place close to the point A or B?

If you think that both bombs fell from the same source and just randomly deviated from each other, then the middle point would be the most dangerous place, because it gives the highest probability that the most probable outcome is in the middle. However, if you think that those two bombs have independent sources, then the middle is the safest place because it is likely that the third bomb is sourced from either of those two. [It is the best guess because we do not have any information at this moment.] Now if you do not know if the sources are independent, somehow dependent or both are the same, then the answer is unclear.

Note that considering both sources sampled completely independent but with the bandwidth deduced from the distance between the points is the usual way for Kernel Density Estimation. The assumption that the sampled points are uncorrelated seems a bit weird, if not illogical at all. There is no relation between making distribution around points smooth and the spatial information. Doing this properly the estimator f(x) would be a sum of delta functions on points. But this is unacceptable, so the estimator is smoothed with Gaussians or some another kernels. Now if the smoothness (bandwidth) is big enough it is possible (depending on the kernel shape) that the middle is less safe than the end points. [For example see how the most dangerous place changes with the bandwidth parameter for the sum of 2 Gaussians.] In other words the probability in the middle adds equally from both sources and it can be greater than maximum probability from the source A plus probability from the source B at the point A.

If the common kernel is taken for both points, then the most probable place for the third point is the middle, which leaves end points safest - with simple shape of the kernel the middle is the most dangerous place and the further you are from the middle the safe you are; regardless whether you stand right inside the shell hole A or B, or next to it.

Personally I would hide in 1/4 of the distance between A and B from the point A or B, i.e. in the middle between the end point A or B and the middle point. What kernel does satisfy this? I do not know.

How I made new Microsoft C++ compiler work without Visual Studio installation

2009-01-28T19:27:00.000-08:00

My task was to make minimalist compiler configuration running on a machine without installed Visual studio. This helps to work on C++ programs on any computer: copy your files, copy the compiler, and then compile and run. Below are the steps I did to make it working.

1. I downloaded and installed free Visual C++ 2008 Express Edition
http://www.microsoft.com/express/download/

2. I created target directory d:\c15 and a file setvars.bat with the following content
set C15=d:\c15
set PATH=%C15%\bin;%PATH%
set INCLUDE=%C15%\include
set LIB=%C15%\lib

3. I copied files

mspdb80.dll
from "Microsoft Visual Studio 9.0\Common7\IDE\" into "d:\c15\bin\"

c1.dll, c1xx.dll, c2.dll, cl.exe, link.exe
from "Microsoft Visual Studio 9.0\VC\bin\" into "d:\c15\bin\"

whole directory 1033
from "Microsoft Visual Studio 9.0\VC\bin\" into "d:\c15\bin\"

whole directory include
from "Microsoft Visual Studio 9.0\VC\" into "d:\c15\"

libcmt.lib, libcpmt.lib, oldnames.lib
from "Microsoft Visual Studio 9.0\VC\lib\" into "d:\c15\lib\"

Kernel32.Lib
from "Microsoft SDKs\Windows\v6.0A\Lib" into "d:\c15\lib\"

4. At this moment I was able to compile C++ programs with the copied compiler. I started cmd, cleaned the PATH variable, run setvars (step 1), and cl.exe compiled the C++ and C programs. But when I moved this directory to the computer which does not have Visual studio installed then cl.exe failed with a message: "The system cannot execute the specified program".

5. I looked into cl.exe binary searching for the string "manifest". That part is XML. Noted the lines
name="Microsoft.VC90.CRT" version="9.0.21022.8"
Note: version reported by cl.exe is different!

6. I went into C:\WINDOWS\WinSxS\Manifests and found file
Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375.manifest
Note: version numbers match!

7. I copied this file into "d:\c15\bin" directory and renamed it into
Microsoft.VC90.CRT.manifest
Note: the name in step 1, the name of the file, and the name inside this file is the same "Microsoft.VC90.CRT"!

8. I went into C:\WINDOWS\WinSxS and found directory
x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375
Note: version match!

9. Copied all dlls from this directory into "d:\c15\bin" directory.

Voila, it worked in Virtual Windows with no Visual Studio installation.

Intelligent design

2008-12-09T20:41:00.000-08:00

Biological life is so complex that it must be created by intelligent design. On the other hand, it is so complex that no sane intelligent creature would dare to engineer it.

Simplistic View On Evolution

2008-12-09T17:49:00.000-08:00

This is the environment.

These are the organisms.

This is the phenotype.

These are mutations. The phenotype does not much change.

Genotype explores the environment, learns about the limits suitable for living. If the environment (including diseases) does not change, the genotype stays in steady state.

The environment changes. Inbreeding occurs (sexual), mutations become obvious, phenotype swells.

Genes are in desperate search for new environment. New exists are found due to the memory obtained when the living conditions were good and stable.

Two incompatible genotypes are now created.

Saccade hypothesis

2008-10-02T00:17:00.000-07:00

Thinking about how the visual cortex works I came to the conclusion that the 'movies' must be produced by the cortex itself. This is to make sure that the changes of the picture are not the changes in the world. One example for this can be saccades. I started looking for more information about this subject and surprisingly found that the main reason for saccades is difference in retina resolution. I do not agree with this.

According to the current belief the main reason for saccades of the human eye is that the central part of the retina, the fovea, plays a critical role in resolving objects. By moving the eye so that small parts of a scene can be sensed with greater resolution, body resources can be used more efficiently.

In my opinion this is not the case. The main reason for saccades is building abstract invariant of the picture. In visual cortex the information propagates up the higher level by building more abstract notation of the picture. The voluntary changing of the picture is the way for the cortex to deduce the common information. The crucial role here is that the brain (cortex) knows that the real cause (objects being seen) does not change. So the cortex must be working differently when the picture coming from the retina changes due to external factors or changes due to internally motivated saccades.

The memory prediction mechanism allows comparing the changed picture with the predicted according to a particular distance and direction of the saccade. This is done in partly learned memory. But the same mechanism works as finding the nvariants in the changing pictures produced by slightly different view.

The analogy of this process is the theory and the experiment. The theory is proven by different experiments giving the different results but fitting in the theory prediction. The assumption is that the essence does not change with the experiments.

"Saccades are a widespread phenomenon across animals with image-forming visual systems. They have been observed in animals across three phyla, including animals that do not have a fovea (most vertebrates do not) and animals that cannot move their eyes independently of their head (such as insects)." [Land, MF. "Motion and vision: why animals move their eyes". J Comp Physiol A. 1999 185:341–352.] Although in this paper the author argues that the reason for saccades "is the need to avoid the blur that results from the long response time of the photoreceptors".

Assuming that building image invariant is the main reason for saccades, one would come to a conclusion that the other voluntary picture changes should happen in the eye. Indeed, some animals allow their eye rotate [above paper]. Why human eyes do not rotate I do not know. Maybe our vision system is complex enough (3D) to allow less emphasis on obtaining picture.

Sensory cortex and saccade motor cortex must be closely coupled. Because the sensory cortex governs the motor cortex and motor cortex has to feed back its lower level information to sensory cortex. There should be many effects which can easily be tested by experiments to support this hypothesis.

How to measure chaos

2008-09-24T18:23:00.000-07:00

This is to propose the way to measure chaos in a computational system.

I studied the reversible algorithms of arithmetic multiplication (factorization problem). I have found a very interesting fact that when the information describing the ensemble grows fast, this exact process is difficult to reverse, i.e. find feasible computation reverse algorithm. Let me explain by example of computation.

Suppose there is an initial set of N bits {x}. If all bits are independent and can initially be set to either 0 or 1, then this is the ensemble of 2^N states. Now suppose we have a function G describing all forbidden states of the ensemble. [The same up to inversion arguments would go with all allowable states.] Initially G=0, since all values of x are allowed.

Next suppose that we have a computational process which is done in atomic steps. Each step can be one of two kinds: either creating a new bit or forgetting an old. For example, add a new bit y₁=x₁&x₂ (let us signify & as AND operation, | as OR, and ^ as XOR). The G function becomes y₁^(x₁&x₂) if previous G=0, or in general case G_next=G_previous|y₁^(x₁&x₂). Forgetting a bit x₂ would make G_next=G_x₂=0&G_x₂=1, where G_x2=0 is G with x₂ replaced with 0.

It seems easy to keep G function defined while doing the calculation process. This is important because while we know G function the calculation is immediately reversible – it is easy at each backward step find out the previous forgotten bit. But the problem appears when G function becomes impractically huge in representation. This happens, for example, with arithmetic multiplication. The only way to attack this problem is to find symmetries in the function so to make it computationally practical, smaller.

Suppose that with a given intellectual strength we find the smallest representation of G function. This function can be written as a sequence of the atomic steps A=B&C, where A,B,C are bits or their negations (say A=B|C is ~A=~B&~C). The negation is not an operation, rather a perception of the bit. With this representation the function becomes measurable in number of atomic computational steps.

That makes it very natural to define the chaos as the increase of the size of the function G. Note that the definition includes the intellectual strength of the observer, which seems natural for the definition of chaos. Also it qualitatively resembles Lyapunov exponents as chaos measure.

Precise theory for Artificial Intelligence

2008-09-14T22:13:00.000-07:00

Every research area has its own language. But there is a distinction between precise and non-precise theories describing the areas of interest. In physics, for example, the language is mathematical formulas. And this is a precise formal language; because formulae help, in general, to solve physical problems even if people applying the formulae have different interpretation of them. We can argue on the definition of Time and Enegry. But when it comes to the formula with E and t, everyone will get the same result regardless of personal (internal) understanding of these concepts.

Completely different situation is with Artificial Intelligence research area. Here are concepts of information, memory, awareness giving completely different philosophical theories depending on how the concepts are being understood. There is so big variety of opinions that it is difficult to call this area of research scientific. Nevertheless some groups of people sharing the same beliefs are able to communicate and even to make some progress in ideas. But with no practical results. Obviously now the understanding of the problem in question is different than ten years ago. Hopefully a simple model can be built in some near future, just to give the common basis for terminology used in this field.

Beating the nature

2008-07-15T12:30:00.000-07:00

We are not cleverer than the nature. Just sometimes the nature does not understand us.

2008-01-08T20:14:00.000-08:00

When we get together our intellect does not add up, but our stupidity does.

Number of features grows logarithmically

2007-12-09T14:43:00.000-08:00

The main problem in big software projects is complexity growing as the project unfolds. Generally when the main architecture is ready and the product has been started the number of architectural features grows with the size of the code. The speed of growing, though, is not constant. At the beginning it is almost linear and equal to time multiplied by team. But the longer it lasts, the bigger size of the code, the more complex is the product to develop further. From my personal experience the time spent to develop a new feature of average size increases by an exponential law. [Features are meant to be architectural but perceived in complexity from the customer point of view.] This idea gives a rule to predict milestones and practical limit of the product. For example, if the first 10 features took 4 days to develop and the second 10 new features took another 5 days to develop, then to implement 50 features will take 7 weeks and to implement 100 features 27 weeks. [Formula used is y=F*ln(kt+1). In the example above we find that k=1/16 and F=10/ln1.25]

On coding practices

2007-09-06T20:00:00.000-07:00

Coding rules are written for particular community of people who intend to review, maintain, or use the code. It is called coding standard. But there are 2 other important sources which influence the style. These are taste and simplicity. I disagree on that the coding standard, whatever it is, has to be followed blindly.

So 3 points influencing the style are:
1. Personal taste: You personal choice
2. Simplicity/clarity: Do not sacrifice clarity to your or other people's taste
3. Community: What most (other) people expect to see when looking at the code.

My personal preferences are:
1. Find which community is likely to work later with the code.
2. Follow coding rules if they do not disagree with your taste and code clarity.
3. If a rule is not good from your point of view, decide which one: your taste or community is more important in that context.
4. If a rule makes code less simple or clear, ignore the rule. [This is very subjective so be prepare to stand for your opinion]

19.04.2007

Console/Window executables

2007-08-28T19:47:00.000-07:00

Simple trick to suppress console window to pop up is to change the flag 03 to 02 as it is done in the following example (in case you forgot 03 is ctrl-C and 02 is ctrl-B):


0000: 4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00  MZ_ ♥   ♦   яя
0010: B8 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00  ё       @
0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0030: 00 00 00 00 00 00 00 00 00 00 00 00 80 00 00 00              _
0040: 0E 1F BA 0E 00 B4 09 CD 21 B8 01 4C CD 21 54 68  ♫▼є♫ _○Н!ё☺LН!Th
0050: 69 73 20 70 72 6F 67 72 61 6D 20 63 61 6E 6E 6F  is program canno
0060: 74 20 62 65 20 72 75 6E 20 69 6E 20 44 4F 53 20  t be run in DOS
0070: 6D 6F 64 65 2E 0D 0D 0A 24 00 00 00 00 00 00 00  mode.♪♪◙$
0080: 50 45 00 00 4C 01 12 00 7B DA D4 46 00 68 5E 00  PE  L☺↕ {ЪФF h^
0090: 34 1F 00 00 E0 00 06 01 0B 01 02 38 00 50 0A 00  4▼  а ♠☺♂☺☻8 P◙
00A0: 00 1C 10 00 00 F8 20 00 A0 12 00 00 00 10 00 00   ∟►  ш   ↕   ►
00B0: 00 60 0A 00 00 00 40 00 00 10 00 00 00 02 00 00   `◙   @  ►   ☻
00C0: 04 00 00 00 01 00 00 00 04 00 00 00 00 00 00 00  ♦   ☺   ♦
00D0: 00 F0 7F 00 00 06 00 00 4A 86 63 00 03 00 00 00   р⌂  ♠  J┼c ♥
00E0: 00 00 20 00 00 10 00 00 00 00 10 00 00 10 00 00       ►    ►  ►
00F0: 00 00 00 00 10 00 00 00 00 90 30 00 22 46 00 00      ►    _0 "F
0100: 00 E0 30 00 98 20 00 00 00 00 00 00 00 00 00 00   а0 _
0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0120: 00 10 31 00 88 4B 00 00 00 00 00 00 00 00 00 00   ►1 _K
0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

This flag is at the end of the line 00D0. This flag is not in absolute offset so you may try to guess.

Software complexity limit

2007-08-14T20:58:00.000-07:00

Software development is a sequential process. The product is always being developed bit by bit. The main problem arises when new bits added to the future product break the previous functionality. This is called complexity limit. Once the product developed reaches its complexity limit no reasonable development becomes possible. The complexity limit is obviously subject to both the design and the process. The difference between them is that if the design adds absolute value in fighting with complexity the process adds relative. That means that the process can leverage the extent of the final product, but cannot define it. As example, suppose that the product is 100 percent functionality and with some given design the complexity limit appears at about 50 percent. Next suppose that the process may leverage the extent by 20 percent of the reached extent. So the final product will have 50+50*0.2 = 60 percent of desired functionality. Now if the design is poor and allows going only up to say 20 percent, then the gain obtained from the process is 20*0.2 = 4 percent (totalling 24). If the design is good allowing 80 percent with no process, then the process will add 16 percent (totalling 96).

Under design I mean here all development solutions on the whole scale from architectural to coding. Under process I mean here the standards accepted by the development team.

Notes on programming

2007-07-25T16:22:00.001-07:00

Programming is a fight against the complexity

When a program is being developed its complexity may increase or remain the same. We can define the program complexity as the complexity of its most complex component (the component decomposition can be seen as a separate component itself). It would be an interesting question what kind of program modifications increase the complexity and what do not.

Complexity multiplies but capacity does not sum up

A number of people cannot create a program which is more complex than at least one of these people can understand. From the other hand, linearly adding a new component of lesser complexity that already exists does not increase the overall complexity. Keyword here is “linearly”, which means that complexity of the decomposition (referring to the level on which the component added) does not increase as well, or increase so as not to outweigh the overall complexity. But making the new component interdependent with existing components opens new dimensions in internal states of the program (non-linearity) that is multiplying the complexity.

Architecture

Each character of the code has its own path to the top level concept of the program. Top level concept of the program lies on the abstract layers braking up the program architecture into independent components, which are lying on lower level layers and so on until we reach each separate character of the code.
Understanding this path from the bottom level to the top level is architectural awareness. It is important that the programmer has one. Otherwise the complexity of the product can grow incredibly fast.

Push, pop, and pull requirements

The requirements evolve from the customer to the final coder. It is not possible to imagine that the programmer can develop a program targeting each customer requirement individually.

The requirements can be generally grouped into 3 categories.

Customer requirements (Pull). Something which defines the product.
Architect requirements (Pop). The model which can be implemented in software and is proven to mimic the customer definition of the product. These requirements are conditions to components and their interaction.
Code requirements (Push). Testable functionality of the program.

Push requirements are satisfied when the program undergoes a number of scenarios defined the requirements. This can be done by auto scripts validating expected output.
Pop requirements are satisfied when the program conforms to the defined component interfaces. This can be done by documented and frozen APIs.
Pull requirements are satisfied automatically given that the model is adequate to the required product. These can be customer acceptance tests, but they would not be a part of the product development.

Failure to break the requirements into these 3 categories and treat the customer requirements as the requirements to the software leads to ad hoc style programming. Programs written in this way are difficult to modify and they are sources of plentiful hard tracking bugs. This can work only for small programs.

Visual Studio 2005 Express, OpenGL and Glut

2007-05-09T19:35:00.000-07:00

Visual Studio 2005 Express does not have SDK. Instead of installing huge SDK what I did to make Visual Studio 2005 Express to run with openGl and Glut is the following.

1. Downloaded and unpacked
glutdlls37beta.zip
and
opengl95.exe
(ftp://ftp.microsoft.com/softlib/mslfiles/opengl95.exe)

2. copied glut32.dll into windows directory

3. created dir GL in include
in my case it was
C:\Program Files\Microsoft Visual Studio 8\VC\include\GL

4. copied *.h files in there

5. copied *.lib files in lib directory
in my case it was in
C:\Program Files\Microsoft Visual Studio 8\VC\lib

Also have a look at:
http://www.opengl.org/resources/faq/technical/gettingstarted.htm
(2.060 What do I need to compile and run OpenGL programs?)