Equation for Radiative Transfer

I have no idea on what to post this week. Not exactly because I do not have a theme, but it is because I could not get even close to understand enough of the subject to post anything very conclusive about it.

As I said before, I have just started taking PhD Classes at INPE. I'm the only students on this cycle (3rd/2008) attending the Computational Optics course. I have decided to go for Applied Computing as the Area for my PhD. I was a little tired on doing research only in Informatics. I'm glad to say that I found an area that I know absolutely nothing about :-).

On these classes, we will study a single equation. Yeap, I also thought it would be easy when the professor mentioned. The trick is that the equation looks like this:

This is the base form of the equation for Radiative Transfer . It is going to be my baby for the next three months. I'm not going to even try explaining it to you. However, its purpose is really nice.

In general, this equation allows scientists to evaluate the interactions of a flow of particles when they move around the space and go from one medium to the other. In my particular case, I'm learning how to understand the interactions that take place when a flow of photons (light) hits the ocean.

Behind the algorithm, there is a very noble cause. By understanding the amount of light that reaches the ocean and the amount of light that returns (refracts) from the ocean, scientists can estimate the characteristics of the water on that particular area. For example, it would be possible to evaluate the amount of chlorophyll dissolved into the water what would allow additional calculation in order to identify how much life (fishes, etc) is also present/supported on such area. This would be a nice feature for Google Earth, isn't it? Checking out the fish populations on the move from one spot of the earth to the other, how cool would it be?

The "real" scientists, not students like me, use the most of the equation. However, I'm glad to say that we have defined several simplifications that will allow even an IT guy like me to understand (or at least try understanding) how to solve the problem. Here are some of the assumed simplifications:

1. The flow of photons will not vary on time. It is the same as assuming that the Sun would always have the same intensity of light, does not matter what time of the day.
2. We are not evaluating a particular spot on the ocean. We assume that we have an infinite geometry. We assume that all photons emitted by the light source will always reach our ocean, does not matter where.
3. Our light source is isotropic; it means that photons are equally emitted in all directions. There is no preference(different orientation) for any single spot.
4. The photons will reach a single surface. In this case, the equation will not consider the flow of photons going through clouds before they reach the ocean, for example.
5. There is also a set of frequencies(light colors) being studied. In our case, we will consider all frequencies of the visual spectrum (red to violet) as a single band. Using the average values for the entire band also makes the equation simpler. We trade accuracy by simplification on this one.
6. We also consider that there is no internal source of light. On another words, we assume that no luminous fish were swimming on our ocean during the satellite scanning.
   

There are a few other simplifications that I cannot really remember now. They make the equation "much" simpler than its base form. Here its simplified version is:



Well, after the initial 30 second that you have been cursing yourself because you have no clue on what this means, I should tell that there are a few methods already tested and that can solve this equation. Here are the ones we will implements as/if time permits:

Method Sn and Method Monte Carlo.

At this point, I'm still working on understanding the problem. I will post something about its resolution as soon as I get there.

-Luciano - Don't worry, I'm completely freaked out too :-)

VMWare ESX 3.5

Hello everyone

Last week, the most of my work was related to getting my lab infrastructure ready for new projects. I have had a few performance problems with VMware Free Server and I was looking for other options.

I was deploying an Interactive Voice Response Unit on the top of a VMware Server. My Server was composed by a Dual Xeon processor with 8 GRAM and about 500G HD. The system used to work just fine with several CTI/CRM suites. However, it seems that my previous real-time applications were not as "real-time" as the IVR is.

It was really annoying to hear the IVR scripts playing that slow. I could hear the prompts when I dialed-in with the phone but it was like a kid telling the words rrreeeaaaallllyyyy sslllloooowwww. This is very likely because of the problem I mentioned in a previous post that even with 8 processors available, only one of them was being used to each of my Virtual Machines.

The good thing is that I was checking out a candidate list of supported platforms for the next releases of our products and I saw that VMware ESXi is one of the options. Even better that VMware just released a free (limited-) version of this software. I have been able to download and start playing with it. Here are some findings from the tests executed last week with VMware ESXi 3.5:

Lab Environment: IBM x3650, 16GRAM, 600HD.

Installation: The ESXi software is distributed as a bootable CD. It is a sort of Linux that installs on the top of the raw equipment. I was not sure if it would work with my equipment since it was brand new for me; however, I had absolutely no problem to make it work. I did not have to enter almost any information during the installation process. Even things like IP addresses had to be configured after the entire system was already deployed. The default installation was enough to bring the entire system up.

Initial Configuration: The very first thing to do with ESXi is getting to its web-based interface. The system is initially configured with Dynamic DHCP. From the console, you should be able to see the IP address and change it to whatever fits into your network. With the new IP defined (or even the old one if you rather use your DHCP), just use you internet browser and get to it. From there you will be able to download the "VMware Infrastructure Client". Just install this on your client machine and you will be ready to go.

The GUI is very nice. It allows you to take a look on all your system resources. I have not tried anything advanced yet. It is useful to check all the system components and - of course - play with your virtual machines.

Importing Virtual Machines: This is not as easy as it sounds. This is an area that I think ESXi still needs lots of improvements. VMware has released a VMware Converter. The concept is great but I could not make it work for Linux Guests. Converter provides a wizard that asks you where the source machine is and to where it is supposed to be transfered. I was very impressed initially because it even offered me to convert directly from my VMWare Server format to the ESXi format. I just selected the source, entered the IP address and credentials of my new ESXi server and the magic started. To be fair, I get to say that the conversion of my Nuance Server which was working on a Windows box got converted and imported to ESXi perfectly. It was really like magic, never did anything easier. The nightmare started when I tried transferring my Redhat 5 system from VMServer to ESXi. I wasted a few hours but nothing worked. Luck me, it was fairly quick to create a new ESXi-based virtual machine and load the RedHat directly into it. Bottom line, converter needs improvements for conversion of Linux boxes. I have tried copying the virtual HDs (vmdk) from one side to the other but I got no luck to make the virtual machines work either.

Storage Devices: ESXi works with its own file system (vmfs3). Based on conversations with some friends, it seems that the management of the file systems on previous releases was really painful. On the 3.5 release, I get to say that I see no problems at all. The GUI really does a great job in order to create folders, move virtual machines from one place to the other, etc. The main issue (or not depending of your expectations) I found was because ESXi does not accept/recognize any kind of USB Storage. I was hoping that I could plug an additional External HD for my Virtual machines but it did not work. The way ESXi manages the disks is via the GUI. You'd have to add a new "Storage" device. The "Storage" device can be iSCSI/SCSI HDs or a Network File System. This may be a little problematic if you are not used to manage UNIX servers.

In my case, I used this as an excuse to install my first Windows 2008 Server. Since I had to find some way to translate my windows shared folder into NFS, I have decided to install windows 2008 in order to use the "Services for UNIX" package. This package is available via download from Microsoft for Win2003/2008 Server. I had Win2003 Server 64-bits and the package is not available for that platform. Since I'd have to go back to a 32-bit platform anyway, I decided to go to win2008 to have some fun.

I have installed the Windows 2008 Data Center - Without Hyper-V edition. It installs just fine (it is windows right :-)). The standard GUI looks a lot like Windows Vista but without all the cool colors. On this new version of windows, you will notice that not all packages and services are installed right away. The main change I saw was that you have tens of optional items to install after the initial deployment. For the "Services for UNIX", you just have to add this "Function" from the File Server tree. In order to have the UNIX IDs synchronized to the Windows Users, you will need to have one Active Directory installed as well. If you did everything on the Windows side correctly, all you have to do is creating a windows NFS share and creating a Storage Device from the ESXi side.

From that point on, ESXi server can access whatever files you have on your NFS share. This is particularly important if you want to access a common repository of CD/DVD images. For CD/DVD images, ESXi-based virtual machines can only see files on a "Client Device"(**) (DVD-ROM on your client computer), Host device (DVD-ROM on your host computer or Data Store ISO Image (CD/DVD Images inside a valid Storage Device).

(**) The "Client Device" mode is supposed to allow you to select a local ISO image as well. However, it did not work for my Client running on a Vista environment, connecting to an ISO image on a shared windows folder. If someone know how to make it work I'd be more than happy to hear it.

Final Comments: I'm not sure if this is important for most of you, but it is for sure important for me. ESXi 3.5 cannot use IDE Hard Drives. Because I use a few simulators that run on the top of old Linux distributions, this is something that prevented me of migrating all my servers to ESXi. Unfortunately, my PBX simulator could not be installed on ESXi because it does not have SCSI drivers. I still have to keep my VMWare Server working for these old Virtual Machines.

I think this is all I can recall from this week. I will post additional information in case I find anything new.

-Luciano