xorg/ GoingModular

Going Modular

This is an outline of a scheme for incrementally migrating the X.org tree from a monolithic build based on imake to a modular build based on automake, autoconf and libtool. The goal is to accurately reproduce the existing binaries as closely as possible.

* Note: This plan was replaced by ModularizationProposal and that migration was finished in 2007, so this page is now historical. *

Goals

The goal of this process is to produce separately distributable “packages” for various parts of the system, including (but not limited to)

  • Individual libraries
  • Individual video drivers
  • Collections of Applications
  • The core X server binary

In my opinion, we should strive to break things up enough so that no distribution feels the need to break things up further. This limits the downstream impact for any given patch. I also believe we should permit either the X server or libraries to be built without requiring the other; this means that all shared include files must be packaged separately.

With this in mind, I believe the correct order of execution is something like:

  1. Move include files.
    1. global includes move to new packages for each sub-system
    2. library-specific headers move into the library in such a way as to not require any additional work before the library itself can be built -- e.g. in Xlib, a subdirectory 'X11' must hold the various Xlib headers so that -I. will permit #include to work.
  2. Fix the 'make includes' pass to actually install the header files in $(DESTDIR)/...
  3. Fix Imakefiles so that these new locations work everywhere. Yes, this is busy work given the goal of eliminating the Imakefiles shortly, but it means that we can still build the tree in the interim.
  4. Autotool the global include packages. We can steal liberally from the debrix and xlibs projects here.
  5. Autotool the libraries, starting at the bottom of the stack and moving upwards. As each is autotooled, the imake bits need to be changed to locate the library in the installed location ($(DESTDIR)/...) and that library should be removed from the imake system.
  6. Select appropriate modularization boundaries for the various X applications, modularizing them on an as-needed basis.
  7. Borrowing liberally from debrix, construct autotool packages for the X server headers.
  8. Convert the server over to use the installed server headers
  9. autotool the X server drivers. Can we do this first?
  10. autotool the X server core. Do we do this first?

Lots of this is just speculation at this point. I think the key points are:

  • Use imake to transition to a modular system; make sure the unfinished parts remain buildable from the monolithic imake system.
  • Leave Imakefiles around in each finished module. This should make 'xmkmf -a' work in a lot of cases.
  • Ensure binary compatibility at each stage by checking the results against the monolithic build. I think the actual binaries will be slightly different as libtool will end up passing different options to ld than imake does. If we could avoid this difference, we could actually compare the modular vs monolithic builds of each piece rather easily.

I'd like to see comments and suggestions for change, and I expect as we follow through this process that we'll have to come back here and make changes, so treat this as a working document, not a finished plan.

-- Main.?KeithPackard - 19 Oct 2004

Questions and Comments

Here are a few questions/comments I have after reading this page:

  • What is the target version of autotools? 2.59 with shiny new features or 2.13 for backwards compatibility with older systems?
  • Autoconf has the ability to configure sub-projects using AC_CONFIG_SUBDIRS. I've found it useful in the past to create a toplevel project that does nothing but configure all the subprojects with identical configure lines, using a shared config.cache file.
  • Reading the above text, it sounds like the idea is to create autotools packages for header files, separate from the code that actually implements each library. Is that the plan, or am I mis-reading the text?
  • For comparing the resulting binaries, I think the easiest way would be to clean up the output of 'objdump -t' for each object and make a note of added/missing symbols or of symbols which have changed size unexpectedly.
  • It isn't stated explicitly above, but I assume the goal is to support separated build directories? It's probably worth stating upfront, since it's quite easy to miss a $(srcdir) somewhere in the Makefile.am which would break this functionality.
  • Most projects seem to have local .m4 files for project-specific things. Would it make sense to have an X.org-wide M4 module so that all projects have the same supporting macros? -- Main.?RayLehtiniemi - 20 Dec 2004