Tuesday, February 03, 2009

A baysian approach to entropy

We'll see if this way of looking at entropy makes it more tangible!

We'll assume that we know all the physical definitions and equations from classical mechanics (or quantum for that matter). Quantities such as energy, volume, velocity, forces, interaction potential are known. While quantities such as pressure, temperature, entropy, free energy are yet to be defined. We'll see why we need these new quantities if we hope to describe the system at hand in the least risky manner.

Consider a box which does not leak energy, through any mechanism. Let the box be filled with marbles in vacuum. Initially the marbles are stationary and their positions are known. Now the box is given to an unknown person who shakes it vigorously and gives it back to you. You now know the total energy content of the box, but since you didn't witness the shaking yourself you are not very sure about the velocity of each particle.

Now it is commons sense that we expect that some fraction of the energy of the box can be transferred into some other system. We do not yet know if this is entirely possible. But we can always hope, because the law of conservation of energy does not prohibit that (this is a classical mechanical law, hence we know its truth).

In a slight detour, let's consider a system where we have a single marble situated at the center of a cubical box. The box is shaken such that the marble is now moving parallel to one of the walls. This is a perfectly deterministic system, the equation of this system can be written as:

f(U,V,N) == 0

where U is the energy of the system (marble), V is the confinement volume and N is the number of marbles (=1). We can bravely say that the system is deterministic. The energy transfer from such a system is going to be easy since we exactly know how it is behaving. This can be generalized to any number of marbles as long as we know exactly the velocity and initial positions of all of them.

The equation of this multiple marble deterministic system will be the usual Hamiltonian equation:

symbolically f(U,V,N,{p_i,q_i}) == 0

Clearly this simple situation differs from our more complicated box with more than 1 marbles. Let {p_i(t),q_i(t)}_n be the set of all possible trajectories of our system. Depending on the initial shaking, the system will choose one of these trajectories. So the actual equation of the system will be

f(U,V,N,{p_i,q_i}_m) == 0

The subscript m corresponds to the shaking procedure employed and will vary depending upon it. Since we do not know how the energy was transferred to the system, we do not know which m to choose. This is repetition of the same statement that we do not know the trajectory of each and every particle. It is intuitive to assume that the maximal amount of energy that can be transferred from our box to any other system is dependent upon the particular trajectories the marbles are following. For a given method, some trajectories will yield higher energy transfer than others.

Now, even if we don't know all information about the system, we are required to estimate the maximal amount of energy transfer that is possible. We have no inclination towards any of the specific trajectories and all of them are . We've ignored (or we are unaware of) the actual state of the system and yet we hope to characterize it. We are not familiar with such lack of knowledge in classical mechanics. Let us hope to quantify this ignorance and call it

Hence instead of the old equation of state, our equation now looks like the following:

f(U,V,N,S) == 0

Notice the absence of a particular trajectory. It is also intuitive that since we do not have access to some of the information about the system. This new approach is going to be an approximate one. We can at the best hope that it applies .

If we further write f(U,V,N,S) == 0 as S == S(U,V,N) we arrive at the first postulate of classical thermodynamics. The postulate of existence of entropy. More properties are boldly ascribed to this entropy function (which it seems to obey! in experiments) in further postulates.


In various good books on thermodynamics, the existence of entropy is either postulated as a mathematical fact or is justified retrospectively as a useful tool. I find this ignorance approach much more realistic and logical since it justifies the postulate for me.


Karthik Shekhar said...

Now that you've written it out, I can understand what you're trying to say a bit more clearly. I still need to think about the physical implications more clearly, but I have one silly question upfront:

Going from equation 3 to equation 4 seems to imply that there exists a fundamental relationship between the entropy as a function of time for a n-body system (non-interacting) and the phase variables of the system (p_i and q_i) as a function of time. Is such a relationship known?

Another comment, which you might consider as an augmentation to your analysis which doesn't add anything new. We know that given the initial position and momenta of a set of non-interacting particles, we can know the state of the system at any future time t. The form of your equation 3 seems to imply that the manner in which the box is shaken up predicates the initial positions and the momenta of the particles in the system at the instant when the shaking is stopped. After this, Hamilton's equations take over to determine the system state till eternity.

Philip Carey said...

For paragraph 2. entropy comes into picture only when we do not know the specific trajectory. If we did, we wouldn't require thermodynamics! So, there is no relationship known. Even when dealing with single particle thermodynamics, it is always attached to a heat bath, the specifics of which remain unclear till we define what heat means.

Plus, Landau and Lipschitz has an amazing discussion on why instantaneous entropy cannot be defined.

I didn't understand paragraph 3.

Karthik Shekhar said...

I simply stated that the information about the specific trajectory that the system takes is stored in the particular set of initial conditions conferred on the system by the shaking process - a particular initial state from a large set of "possible initial states" consistent with the values of U,V,N.

It was an assertion (you might think it tautological), that's all.