Projects
Listed below are some of the projects I have worked on in the past. Please realize that I may not be able to talk about some of these in greater detail due to the IP rights of my present or past employer(s).
In particular, this page does not list any projects related to my current job.
- A Virtualized Solaris Operating System with Quality of Service
- Resource Management for Quality of Service in a Custom Operating System
- Quality of Service Support for Legacy Applications
- Signaled Receiver Processing
- An Audit and System Call Interception Mechanism for the Linux Kernel
- Licensing and Security for Commercial Software
- Automation of OS Installation
- A Linux/Unix Platform for Implementing and Deploying Arbitrary Web Services
- Embedded software for environments such as the Sega Dreamcast, the Nintendo Gameboy Advance, the RioCar MP3 player etc.
- Advanced Programming in the Mac OS X Environment
- Hanoimania - Myriad Implementations of the Towers of Hanoi
- [old] Selected School Projects
- Mac Futureware
A Virtualized Solaris Operating System with Quality of Service
I created (researched, designed, and implemented) a "virtualized" version of Sun's Solaris operating system. The idea is to divide the operating system (by creating a software layer in the kernel) into multiple virtual environments, where each virtual OS is capable of running arbitrarily complicated existing applications unmodified. Such a complicated application (Oracle, for example) would typically exercise most components/subsystems of the OS. Resources are made available to a virtual instance with Quality of Service. Moreover, applications in one virtual OS instance are in complete isolation from applications in other instances on the same "real" machine.
Each virtual instance can be managed (administered, configured, rebooted, shutdown etc.) completely independently of the others, and is visible as the "normal" operating system to applications within it. Note that this is different from an emulator: there is only one instance of the OS kernel, but the APIs have been virtualized in the kernel. This results in a much higher virtual instance performance than would be possible with an emulator.
Specific virtualization components include (but are not limited to):
- Virtual system calls
- Virtual uid 0 (each instance has its own "root" user)
- Fair share network scheduler
- Per-virtual OS resource limits on memory, CPU, and link
- Virtual sockets and TLI (including the sockets port space)
- Virtual IP address space
- Virtual NFS
- Virtual disk driver and enhanced VFS (each instance sees its own physical disk that may be resized dynamically, and partitioned as usual)
- Virtual System V IPC layer (each instance gets its own IPC namespace)
- Virtual /dev/kmem (each instance can access
/dev/kmemappropriately without compromising other instances, or the system in general) - Virtual /proc filesystem (each instance gets its own
/procwith only its processes showing up) - Virtual syslog facility
- Virtual device filesystem
- Overall system management layer
Note that this was product quality software and all work was done without ever having seen the source code for Solaris (which obviously is proprietary to Sun, and their source license had enough "wrong" strings attached from my company's point of view).
Note: I have been asked sometimes how this virtualization project (referred to as "V" from now on) relates to, or is different from the upcoming "Zones" feature in Solaris 10, or the FreeBSD "jail" subsystem. Here are some thoughts on this (assuming the reader is familiar with Solaris Zones and FreeBSD "jail"):
- "V"'s goals are different (loftier, in many ways) from the others: it strives to give you the benefits of an OS emulator (or a real OS running on a hardware platform emulator) with far less overhead. Like others, it provides you isolated environments in which you can securely run applications, but unlike others, its isolated environments are very much like the full-blown underlying OS. For example, a virtual environment in "V" comes up just like a normal system (its own
initand startup scripts). Having said that, "Zones" does appear to be very similar. - "V" lets you install and run most applications within a virtual instance, except those that access hardware directly. Since there is a virtual disk driver, applications that want to access disk(s) directly are allowed. The capacity of this virtual disk can be altered dynamically, even if there is a filesystem on it.
- Like others (to varying degrees), "V" lets you assign resource limits (on CPU, memory, disk space, network bandwidth etc.) on each virtual instance.
- "V" is implemented as a set of loadable kernel modules, without referring to the kernel source. It can be dynamically introduced into a running system.
- The project was started in 1999 (on Solaris 7, later carried over to Solaris 8) and was largely done in 2000. At that time, there was no talk of Solaris "Zones" and "jail" was not as mature as it is today. It makes me happy to see Sun heading in a similar direction today with "Zones" (they do have the kernel source!)
- A Solaris Zone cannot be an NFS server, while a "V" instance could.
- A Solaris Zone does not allow the
mknodsystem call, while "V" did. - There are most likely features in Zones that "V" did not.
Resource Management for Quality of Service in a Custom Operating System
I worked on introducing Resource Management for Quality of Service in a custom Operating System derived from FreeBSD. This included work on schedulers for CPU, network, and disk, a pseudo filesystem based resource management API, and a resource management layer to provide seamless quality of service to legacy, unmodified applications.
Quality of Service Support for Legacy Applications
While working on a FreeBSD derived Operating System with Quality of Service (QoS), I extended various protocols like NFS, HTTP, and FTP to incorporate QoS. This was done in such a way that legacy user applications could benefit from QoS without having to be modified in any way.
Protocol processing of received packets in BSD Unix is interrupt-driven and may cause scheduling anomalies that are unacceptable in systems that provide Quality of Service (QoS) guarantees. I worked on an alternate mechanism, Signaled Receiver Processing (SRP), that generates a signal to the receiving process when a packet arrives. The default action of this signal is to perform protocol processing asynchronously. However, a receiving process may catch, block or ignore the signal and defer protocol processing until a subsequent receive call. In any case, protocol processing occurs in the context of the receiving process and is correctly charged. Therefore, SRP allows the system to enforce and honor QoS guarantees.
An Audit and System Call Interception Mechanism for the Linux Kernel
I designed and implemented a dynamically loadable system call audit mechanism for the Linux kernel. Using this mechanism, it is possible to arrange for arbitrary context sensitive operations to happen upon execution of a system call. For example, a set of system calls can be denied to certain users. Arbitrary system time can be returned to specific applications (for testing time related bugs, say, like the erstwhile Y2K). It is also possible to associate the execution of certain user level programs when certain actions occur. For example, users can associate passwords with files, etc.
Licensing and Security for Commercial Software
I researched and designed ways to secure (commercial) software, and mechanisms to implement licensing schemes.
I extended the "kick-start" installation procedure of Red Hat Linux to simplify it, and to support more installation methods. I designed a hands-off installation scheme for x86 systems wherein certain boot code can be placed in the PROM of the network card, and upon booting, the user is given the option of either installing an OS on the computer, run a minimal OS in RAM, or boot entirely off the network with an NFS root filesystem.
A Linux/Unix Platform for Implementing and Deploying Arbitrary Web Services
This is a highly extensible system for implementing, deploying, and managing web services and their APIs. I envisioned, designed, and implemented this system from scratch (and led the project's further development and management later). Its key components and features include:
- Object-oriented Plug-in Architecture: The system is modular where new services can be plugged-in. The plug-ins themselves are dynamically loaded in a fine grained manner.
- N-Level Recursive: The system can hierarchically instantiate itself, thereby enabling "reseller" support, with each reseller level being arbitrarily customizable and configurable.
- Enterprise Database Support: A database abstraction layer makes the system capable of using most of the major databases. The database to use is selected through a single setting. Configurations for IBM DB2, Oracle, and Postgres were bundled.
- Platform Independence: The system is implemented entirely in Perl, and is capable of functioning fully with no use of platform dependent features of the language.
- Well-defined APIs: The system's set of APIs is exported via multiple mechanisms: Web Services (SOAP), XML-RPC over HTTP/HTTPS and raw XML over SSH (primarily for legacy support). The system allows easy definition of custom APIs through powerful XML libraries built on top of a DOM parser.
- Error Subsystem: The exception based error subsystem collects all errors encountered along with extensive context information, and is capable of reporting the information in a detailed debugger-style stack-trace, presented as one or more of the following: pre-formatted text, XML (for parsing by a GUI, say) and pre-formatted HTML (for inclusion by a GUI, say).
- The Registry: This module consists of a configuration and settings manager, and an extensible validation library. This provides a flexible way to store, retrieve, and validate arbitrarily complicated data. The validation library can perform various kinds of checks on arbitrarily typed data (like "email address list", "list of FQDNs", etc.). The module employs shared memory for better performance and scalability on systems that support System V IPC.
- The Auth Module: This module provides various methods of authorizing and authenticating requests.
- Internationalization Support: All messages in the system are internationalized.
- Black Box Clients: The system provides black box clients on various platforms (Linux, Solaris, Windows 2000 etc.) for the APIs it exports. These clients are self-contained, and stream XML over their standard input/output.
I created software for various embedded environments, particularly as part of the Towers Of Hanoi project. The environments include the Sega Dreamcast Game Console, the Nintendo Gameboy Advance Hand-held System, the RioCar MP3 Player, and the ProGear WebPad. My other related experiments include running TCP/IP over Infra-red and populating a Debian Linux distribution on an ARM-based embedded system (for on-board software development), running 5th edition UNIX on the Nintendo Game Boy Advance, and numerous miscellaneous projects involving MIPS and x86 development boards.
Advanced Programming in the Mac OS X Environment
Here is a semi-organized account of some of the things I have done with and on Mac OS X.
I created myriad implementations of the Towers Of Hanoi. Please go to the Hanoimania page for details.
Here is a list of selected school projects that I did during my undergraduate days.
Here is a list of some "futureware" (similar to "vaporware") projects that I have in mind for Mac OS X.