Saturday, April 19, 2008

Distributed Compile and Caching

This trick has boosted my compilation several times faster now. Here is the steps:

  1. First, you need to install both ccache and icecream (search on the Internet!). The default place for icecream files are under /opt/icecream.
  2. Create a folder /opt/ccache and /opt/ccache/bin
  3. Create file named "gcc" and type:
  4. #! /bin/sh
    export CCACHE_PATH=/opt/icecream/bin
    export PATH=/opt/icecream/bin:/usr/bin:$PATH
    ccache gcc "$@"
  5. Repeat step 3 and 4, but change line "ccache gcc" to "ccache c++" and name the file c++.
  6. Do the same thing (step 5) for cc abd g++
  7. Modify path (I put this in my ~/.profile): export PATH=/opt/ccache/bin:$PATH


To start icecream, as root do this: /etc/init.d/icecream restart
This will start the icecc daemon (iceccd). Here is my processes:

> ps -ef | grep ice
icecream 3944 1 0 19:33 ? 00:00:03 /usr/sbin/scheduler -d -l /var/log/icecc_scheduler -n icecream -vv
root 3957 1 0 19:33 ? 00:00:06 /usr/sbin/iceccd -d -l /var/log/iceccd --nice 5 -s linux-hp -n icecream -u icecream -b /var/cache/icecream -vv


(Modify file /etc/sysconfig/icecream as you needed (especially the scheduler and netname). I still not able to make my other computer to participate in the parallel compile (it is separated by a wireless router but in the same IP subnet). I don't know the root cause yet (the log said it was not able to find the scheduler, although I explicitly specify it [not thru broadcast]).

NVIDIA driver on Linux Kernel 2.6.25

The current Nvidia driver (NVIDIA-Linux-x86_64-169.12-pkg2.run) is somehow broken with Linux kernel 2.6.25. I googled and found discussion about this. Some people suggested to replace EXPORT_UNUSED_SYMBOL(init_mm) to EXPORT_SYMBOL(init_mm) in
/usr/src/linux/arch/x86/kernel/init_task.c. It didn't work (at least on my machine).

I look at the error log in /var/log/nvidia-installer.log. The last few lines in the file told me the global_flush_tlb() was implicitly declared (meaning, it was no longer exported by the Linux kernel. People in the discussion forum mentioned since 2.6.24 or something) while trying to compile nv-vm.c. This look weird, so I extracted the Nvidia driver with the following command:

sh download/nvidia/NVIDIA-Linux-x86_64-169.12-pkg2.run -a -q -n -x

This command extracted both binary files and its source codes onto NVIDIA-Linux-x86_64-169.12-pkg2/
I then cd to NVIDIA-Linux-x86_64-169.12-pkg2/usr/src/nv and commented out calling to that procedure. Here's my modified procedure of that file:

static void nv_flush_caches(void)
{
#if defined(KERNEL_2_4)
// for 2.4 kernels, just automatically flush the caches and invalidate tlbs
nv_execute_on_all_cpus(cache_flush, NULL);
#else
// for 2.6 (and later) kernels, rely on global_flush_tlb
#if defined(NV_CPA_NEEDS_FLUSHING)
nv_execute_on_all_cpus(cache_flush, NULL);
#endif
#if defined (NVCPU_X86) || defined (NVCPU_X86_64)
//global_flush_tlb();
#endif
nv_ext_flush_caches(); // handle other platform flushes if present
#endif
}

I then manually compile the code usual way (make module) from that folder and did "make install" (executed as root). For the binary files, I just manually copied them to their proper places.

Nervously, I typed "startx" and voila....my Xwindows started successfuly! So far, I haven't noticed any anomalies nor problems (this blog is actually done from my machine running the new driver). Apparently, the latest kernel doesn't require flushing cache anymore (it must be done automatically elsewhere).

Thursday, April 17, 2008

Virtualization Era!

Rebooting a machine to switch to another OS on multi-operating-system machine sounds out dated now. With recent VT-enabled technology from both Intel and AMD on certain x86 CPUs, virtualization goes smoother and better.

I just tested Microsoft Virtual PC which enable me to install multiple windows or even Linux under my Windows XP host OS. There are other Virtualization software available, either freeware or not. Here I am trying to list some of them:

Non-free ones:
VMWare: https://www.vmware.com/tryvmware/?a=l&eval=workstation-l
Microsoft Virtual PC: http://www.microsoft.com/windowsxp/virtualpc/
Novel SUSE Linux Exterprise VM Driverpack: http://www.novell.com/products/vmdriverpack/

Freeware versions:
Bochs: http://bochs.sourceforge.net/
QEMU: http://fabrice.bellard.free.fr/qemu/
VirtualBox: http://www.virtualbox.de/ (this software has dual licenses: free and proprietary)


While, to just run windows application on Linux, we can use wine or its sibling, the commercialized one: CrossOver for Linux: http://www.codeweavers.com/products/cxlinux/

To learn more about virtualization and/or virtual machine (don't get confused with Java VM as it's not a system VM): http://en.wikipedia.org/wiki/Virtualization

The more complete list of VM packages:
http://en.wikipedia.org/wiki/Comparison_of_virtual_machines

I will report further once I try some of them. In most cases, the host OS will be OpenSUSE 64-bit ver 10.3 with guest OSs will be: Vista Home Premium SP1, Vista Ultimate 64-bit, Mac OSX 10.4 Tiger and some variants of Linux [Redhat, UBuntu, Debian]. The host system will be my desktop CPU: Intel Quad-core 2.4 GHz, 4 GB RAM, with almost 768 GB total space.

x86 Technical Info

Some very interesting technical information related to x86 (besides, of course, Intel/AMD official sites and Micro$oft):

http://www.sandpile.org
http://www.mgreene.org/wikka/LdrNotes

Wednesday, April 16, 2008

HP 1020 on Linux is now working!

Finally, I am able to print to HP LaserJet 1020 from my Linux machine. Thanks to the guys at http://foo2zjs.rkkda.com/.

Just follow the instructions in the INSTALL file, you should be able to successfuly print. One thing I was unaware was to control CUPS via web (in my case, browse to http://localhost:631). From there, I could then test.  Although I've tested remote printing, it shouldn't be a problem because CUPS will handle all!

Great!!