Security in ACS

Chrooted AOLserver

Setting up the chroot directory

cd /home/aolserver
mkdir bin dev etc tmp usr var

# Create some device files necessary for Oracle.
# (The following is Solaris specific.)
cd dev
mknod kmem c 13 1
mknod mem c 13 0
mknod zero c 13 12
mknod null c 13 2
chmod a+w zero

# Copy necessary files to /etc.
cd ../etc
cp /etc/group /etc/hosts /etc/nsswitch.conf /etc/resolv.conf /etc/netconfig .
grep nsadmin /etc/passwd > passwd

# Create a symlink as /home/aolserver, so we don't have to edit all our
# .ini files.
cd ..
mkdir home
ln -s . home/aolserver

# Make tmp directory world writable.
chmod 1777 tmp 
mkdir var/tmp
chmod 1777 var/tmp

# Copy rm to bin.
cp /bin/rm bin

# Copy unzip to usr/bin.
mkdir usr/bin
cp /usr/bin/unzip usr/bin

# Copy shared libraries to usr.
# Note that it may be desirable to recompile
# with static linked binaries to avoid having
# these libraries in the chroot environment.
# See dynamic_linking_notes
# to get a more accurate list of files needed.
mkdir usr/lib
cp /usr/lib/*.a /usr/lib/*.so.* usr/lib

# If using the ecommerce module with ImageMagick to do image resizing,
# copy ImageMagick files.
mkdirhier usr/local/bin
mkdirhier usr/local/lib
cp /usr/local/bin/convert usr/local/bin
cp /usr/local/lib/ImageMagick* usr/local/lib
mkdirhier usr/openwin/lib
cp /usr/openwin/lib/libdpstk.so.5 usr/openwin/lib
cp /usr/openwin/lib/libdps.so.5 usr/openwin/lib
cp /usr/openwin/lib/libdpstk.so usr/openwin/lib
cp /usr/openwin/lib/libtif.so usr/openwin/lib

# Copy timezone files.
mkdirhier usr/share/lib
cp -r /usr/share/lib/zoneinfo usr/share/lib

# The page root must also be within the chroot environment.
mkdir web
mv /web/servername /home/aolserver/web

# (Optional) make a link to the chrooted pages from the old web directory
ln -s /home/aolserver/web/servername /web/servername

# Copy necessary Oracle files to new root.
mkdirhier /home/aolserver$ORACLE_HOME
cd /home/aolserver$ORACLE_HOME
(cd $ORACLE_HOME; tar cf - bin dbs lib network ocommon rdbms) | tar xvf -

Setting up Oracle

One workaround for this is to connect to Oracle through a TCP connection. This is by far the easiest to set up. The downside is that there is some performance loss going through TCP instead of using IPC. To do this, edit $ORACLE_HOME/network/admin/listener.ora to add a TCP listener and $ORACLE_HOME/network/admin/tnsnames.ora to add a network alias for that listener (see the Net8 Administrator's Guide, or just use netasst). Then have AOLserver use it by putting the network alias as the Datasource entry for the connection pool in your server's .ini file.

If you insist on using IPC, you must configure the database to run in multi-threaded server (MTS) mode. Configuring MTS mode can be somewhat tricky (see this 8.1.5 doc and this 8.1.6 doc). In brief, you must:

• add the following to your initSID.ora file:

  # Configure MTS for IPC connections and start up one server.
  mts_dispatchers = "(PROTOCOL=IPC)(DISP=1)(mul=OFF)(pool=OFF)(PRES=TTC)"
  mts_max_dispatchers = 5
  mts_servers = 1
  mts_max_servers = 20
  

• Make sure there is an IPC listener configured in listener.ora:

  LISTENER =
    (DESCRIPTION_LIST =
      (DESCRIPTION =
        (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC))
      )
    )
  
  SID_LIST_LISTENER =
    (SID_LIST =
      (SID_DESC = 
        (SID_NAME = PLSExtProc) 
        (ORACLE_HOME = /ora8/m01/app/oracle/product/8.1.6) 
        (PROGRAM = extproc) 
      ) 
      (SID_DESC = 
        (GLOBAL_DBNAME = ora8) 
        (ORACLE_HOME = /ora8/m01/app/oracle/product/8.1.6) 
        (SID_NAME = ora8) 
      ) 
    )
  

• add a network alias to tnsnames.ora:

  ORA8_IPC =
    (DESCRIPTION =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC))
      )
      (CONNECT_DATA =
        (SERVICE_NAME = ora8)
        (SRVR = SHARED)
      )
    )
  

• and use that network alias as the datasource in your server's .ini file.

To put Oracle into MTS mode, you must now restart the Oracle server. The listener should be started before the server so that the server can register itself properly with the listener. To verify that Oracle is in MTS mode, connect to Oracle using "sqlplus username/password@ora8_ipc" (substitute the network alias you put in tnsnames.ora for ora8_ipc), and run this SQL statement: select username, program, server from v$session where audsid=userenv('sessionid');. It should return "SHARED" in the SERVER column. If it says "DEDICATED" instead, your server is not in MTS mode.

One last problem with running ACS in a chrooted environment is that Oracle uses Unix domain socket files for IPC that are created in /var/tmp/.oracle. We must replace /var/tmp/.oracle with a symlink to a directory underneath the chroot directory. This must only be done with Oracle shut down!

cd /home/aolserver
mkdir var/tmp/.oracle
chown oracle var/tmp/.oracle
chmod 777 var/tmp/.oracle
# Make sure Oracle is not running before you do this next step!
rm -r /var/tmp/.oracle
ln -s /home/aolserver/var/tmp/.oracle /var/tmp/.oracle

A caveat about specifying directories in .ini files: every path must be relative to the chroot directory (e.g., /home/nsadmin/foo/bar -> /foo/bar), except for AuxConfigdir, which must be an absolute path.

Running AOLserver

The way to get around this problem is to move nsd8x, nsd-oracle, and the server.ini file outside of the chroot environment, and run it using /somewhere/not/in/chroot/nsd-oracle -ikc /somewhere/not/in/chroot/servername.ini -r /home/aolserver from inittab, where nsd-oracle calls /somewhere/not/in/chroot/nsd8x instead of the one within the chroot environment.

Running ACS

• top needs to be installed for admin/monitoring/top
  Note that we do not recommend installing top in chroot. Recent versions of top require the proc filesystem in the chroot environment to run, which is the sort of security risk that you are trying to avoid with chroot. You could use an older version of top based on kmem (since kmem is unfortunately needed for our web serving), or you could write a simple script to cat top (running outside the webserver's chroot jail) into a text file (inside the chroot jail) regularly and just display that text file from the monitoring page.
• ImageMagick is required for ecommerce - see earlier file copying notes
• perl is needed for watchdog
  
  This can be addressed in two ways:
  • Install perl and all associated library files in the chroot directory structure
  This is not recommended - PERL is rather dangerous to be having in a chroot environment. If you do decide to go this route, be sure to manually enter the chroot environment and test the script out to be sure that you have installed all of the files needed for PERL.
  • Compile aolserver-restart.pl with perlcc. This is the preferred method, but will be more difficult than just installing PERL. See static linking notes for more information
  If you do not compile aolserver-errors or install PERL in the chroot environment, be sure to shut off watchdog by setting WatchDogFrequency=0 in your parameters/yourservername.ini file. Otherwise processes will collect every time the server tries to run watchdog.
• qmail is needed for ecommerce
• ispell
• CVS and ssh for the version control module

Disk Issues

Static Linking Notes

• First, we need to determine if your PERL is dynamically or statically linked.
  • Go to your server's /bin directory.
  • Run perlcc -sav aolserver-errors.pl. Examine the output. If you don't see a reference to the file "Fcntl.so", your PERL is probably statically linked: You should be able to cut-and-paste the last gcc command perlcc used, add "-static" to the end, and have a static aolserver-errors binary.
  • If you do see a reference to "Fcntl.so" when running perlcc, your PERL is probably dynamically linked. You'll need to install another copy of PERL that is statically linked and compile with that (or, you could live with having shared libraries in your chroot environment - see dynamic linking notes.)
• If your PERL is dynamically linked, you'll need to install a statically linked PERL:
  • get a copy of perl5 (wget http://www.perl.com/CPAN-local/src/5.0/perl5.005_03.tar.gz) and extract it.
  • Change into your new perl5 directory and read the INSTALL file.
  • Read the appropriate file in the hints subdirectory for your operating system.
  • Run the Configure program, specifying "-Dusedl=n" for static linking (along with whatever other parameters the hints file recommended) (for our Solaris 2.7 (SunOS 5.7) systems, we used sh Configure -Dcc='gcc -B/usr/ccs/bin/' -Dusedl=n -Dprefix=/testroot/perl5 -Uusemymalloc)
  (any directory you have write access to will do for the -Dprefix parameter)
  • Run make, make test, and make install. Watch for errors. Read the INSTALL, README, and hints files again if you have errors.
• Maybe having a statically linked PERL is enough - try running the path/perlcc -sav aolserver-errors.pl command again, (be sure to specify the path to your new statically linked PERL, .ie. /testroot/perl5/bin/perlcc aolserver-errors.pl. You may want to temporarily remove your normal PERL directory from your path to be sure you don't accidentally use it), cut-and-paste the gcc command, and add "-static" as before.
• On a Solaris system, this will probably fail to link. Solaris does not support static linking of a few functions. Work around this by including this fake_dl.c file in the subdirectory and adding it after aolserver-errors.c on the gcc command line. This will replace the dlopen, dlclose, dlsym, and dlerror functions with simple stubs - enough to link, but not enough to let a program work if it relied on those functions (specifically /etc/nsswitch.conf, such as getYYYbyZZZ, getpwYYY, etc) - but luckily our aolserver-errors program does not rely on these.
• Now you should have a statically linked aolserver-errors executable. Feel free to delete your statically linked PERL, you won't need it to run aolserver-errors.
• Final verification: Manually test by chrooting w/ your webserver directory (the command will be something like sudo chroot /home/aolserver /bin/sh, but the directory, location of sh, and whether you need to use sudo could vary depending on your system). Enter the aolserver-errors command from your parameters.ini file (with full path). If you get no error message, watchdog should be ready to go chrooted.

Dynamic Linking Notes

• For every binary file in your chroot environment, run it from within a chrooted shell, look for errors, copy the missing .so file into the chroot environment, and repeat until the program works. This is a slightly inferior method.
• For every binary file in your chroot environment, run ldd [filename] and note the .so files listed. Make sure that these .so files are in the chroot environment. Repeat for the .so files, making sure any .so files they need are also in the chroot environment. You can feel like a smarter better Unix user for knowing how to use the list dynamic dependencies tool.
• Note that you should not need perl to be in the chroot environment even if it's dynamically linked - you can compile your perl programs with perlcc, and just keep the libraries in the chroot jail. This is definitely better than providing access to perl, although it's not as ideal as having a statically linked program. But try running them from within a chrooted shell when you're done anyway, just to be sure that you got everything.