X-Git-Url: https://arthur.barton.de/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=doc%2FDEVELOPER;h=d0f8b5b58eb4dd51902d3d7d6d48ab1b7e6e933e;hb=9528990149433080bcd6c206d8c66f18b5523ee3;hp=c8d4efa48397c4daa36031ed27a279386b15cdde;hpb=275550d63f76fdfda07b756cbb88d5dbadb5f017;p=netatalk.git diff --git a/doc/DEVELOPER b/doc/DEVELOPER index c8d4efa4..d0f8b5b5 100644 --- a/doc/DEVELOPER +++ b/doc/DEVELOPER @@ -1,41 +1,27 @@ -Information for Netatalk 1.5 Developers -======================================= - -For basic installation instructions, see the INSTALL file. - - -netatalk is an implementation of the AppleTalk Protocol Suite. The -current release contains support for EtherTalk Phase I and II, DDP, -RTMP, NBP, ZIP, AEP, ATP, PAP, ASP, and AFP. The complete stack looks -like this on a BSD-derived system: - - AFP - | - ASP PAP - \ / - ATP RTMP NBP ZIP AEP - | | | | | - -+---------------------------------------------------+- (kernel boundary) - | Socket | - +-----------------------+------------+--------------+ - | | TCP | UDP | - | DDP +------------+--------------+ - | | IP | - +-----------------------+---------------------------+ - | Network-Interface | - +---------------------------------------------------+ - -DDP is in the kernel. "atalkd" implements RTMP, NBP, ZIP, and AEP. It -is the AppleTalk equivalent of Unix "routed". There is also a -client-stub library for NBP. ATP and ASP are implemented as -libraries. "papd" allows Macs to spool to "lpd", and "pap" allows Unix -machines to print to AppleTalk connected printers. "psf" is a -PostScript printer filter for "lpd", designed to use "pap". "psorder" -is a PostScript reverser, called by "psf" to reverse pages printed to -face-up stacking printers. "afpd" provides Macs with an interface to -the Unix file system. Refer to the appropriate man pages for -operational information. +Information for Netatalk Developers +=================================== +For basic installation instructions, see http://netatalk.sourceforge.net . + +Netatalk is an implementation of "AFP over TCP". +DSI is a session layer used to carry AFP over TCP. +The complete stack looks like this: + + AFP + | + DSI + | + | (port:548) + | + -+---------------------------+- (kernel boundary) + | Socket | + +------------+--------------+ + | TCP | UDP | + +------------+--------------+ + | IP v4 or v6 | + +---------------------------+ + | Network-Interface | + +---------------------------+ Compilation =========== @@ -67,10 +53,7 @@ Libtool encapsulates the platform specific dependencies for the creation of libraries. It determines if the local platform can support shared libraries or if it only supports static libraries. -Netatalk currently requires libtool 1.4 or higher (1.4b for OpenBSD). - Documentation: http://www.gnu.org/software/libtool/ -Program: (see the GNU mirrors) /gnu/libtool/libtool-1.4.tar.gz 2. GNU m4 GNU m4 is an implementation of the Unix macro processor. It reads @@ -78,24 +61,32 @@ stdin and copies to stdout expanding defined macros as it processes the text. Documentation: http://www.gnu.org/software/m4/ -Program: (see the GNU mirrors) /gnu/m4/m4-1.4.tar.gz 3. Autoconf Autoconf is a package of m4 macros that produce shell scripts to configure source code packages. Documentation: http://www.gnu.org/software/autoconf/ -Program: (see the GNU mirrors) /gnu/autoconf/autoconf-2.52.tar.gz 4. Automake Automake is a tool that generates 'Makefile.in' files. Documentation: http://www.gnu.org/software/automake/ -Program: (see the GNU mirrors) /gnu/automake/automake-1.5.tar.gz + +Required +======== +5. Berkeley DB +Berkeley DB is a programmatic toolkit that provides fast, reliable, +scalable, and mission-critical database support to software +developers. BDB can downloaded from +http://www.oracle.com/database/berkeley-db/index.html +Netatalk's CNID database uses the library and header files from BDB. +Currently, Netatalk supports BDB 4.6 and later. + Optional ======== -5. OpenSSL +6. OpenSSL and/or Libgcrypt The OpenSSL Project is a collaborative effort to develop a robust, commercial-grade, full-featured, and Open Source toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS @@ -103,25 +94,248 @@ v1) protocols as well as a full-strength general purpose cryptography library. This is required to enable DHX login support. -Get everything at http://www.openssl.org/ +Get everything at http://www.openssl.org/ + +The Libgcrypt is a general purpose cryptographic library based on +the code from GnuPG. +This is required to enable DHX2 login support. + +Get everything at http://directory.fsf.org/project/libgcrypt/ -6. TCP Wrappers +7. TCP Wrappers Wietse Venema's network logger, also known as TCPD or LOG_TCP. These programs log the client host name of incoming telnet, ftp, rsh, rlogin, finger etc. requests. Security options are: access control per host, domain and/or service; detection of host name spoofing or host address spoofing; booby traps to implement an early-warning system. - TCP Wrappers can be gotten at ftp://ftp.porcupine.org/pub/security/ +Netatalk uses TCP Wrappers to authorize host access when using +afpovertcp. It should be noted that if DDP is in use, the connection +will still be allowed as TCP Wrappers do not impact DDP connections. -7. PAM (Pluggable Authentication Modules for Linux) -Linux-PAM provides a flexible mechanism for authenticating +8. PAM (Pluggable Authentication Modules) +PAM provides a flexible mechanism for authenticating users. PAM was invented by SUN Microsystems. Author: Andrew Morgan Linux-PAM is a suite of shared libraries that enable the local system administrator to choose how applications authenticate users. - You can get the Linux PAM documentation and sources from http://www.kernel.org/pub/linux/libs/pam/ +Netatalk also supports other standard PAM implementations such as OpenPAM. + +Error checking and logging +========================== +We wan't rigid error checking and concise log messages. This often leads +to signifant code bloat where the relevant function call is buried in error +checking and logging statements. +In order to alleviate error checking and code readability, we provide a set +of error checking macros in . These macros compare the return +value of statements againt 0, NULL, -1 (and maybe more, check it out). +Every macro comes in four flavours: EC_CHECK, EC_CHECK_LOG, EC_CHECK_LOG_ERR +and EC_CHECK_CUSTOM: +- EC_CHECK just checks the CHECK +- EC_CHECK_LOG additionally logs the stringified function call. +- EC_CHECK_LOG_ERR allows specifying the return value +- EC_CHECK_CUSTOM allows custom actions +The macros EC_CHECK* unconditionally jump to a cleanup label where the +neccessary cleanup can be done alongside controlling the return value. +EC_CHECK_CUSTOM doesn't do that, so an extra "goto EC_CLEANUP" may be +performed as appropiate. + +Example: +- stat() without EC macro: + static int func(const char *name) { + int ret = 0; + ... + if ((ret = stat(name, &some_struct_stat)) != 0) { + LOG(...); + ret = -1; /* often needed to explicitly set the error indicating return value */ + goto cleanup; + } + + return ret; + + cleanup: + ... + return ret; + } + +- stat() with EC macro: + static int func(const char *name) { + EC_INIT; /* expands to int ret = 0; */ + + char *uppername = NULL + EC_NULL(uppername = strdup(name)); + EC_ZERO(strtoupper(uppername)); + + EC_ZERO(stat(uppername, &some_struct_stat)); /* expands to complete if block from above */ + + EC_STATUS(0); + +EC_CLEANUP: + if (uppername) free(uppername); + EC_EXIT; + } + +A boileplate function template is: + +int func(void) +{ + EC_INIT; + + ...your code here... + + EC_STATUS(0); + +EC_CLEANUP: + EC_EXIT; +} + +Ini Parser +========== + +The ini parser is taken from . +It has been slightly modified: +- case-sensitive +- "include" directive added +- atalk_iniparser_getstrdup() to complemnt atalk_iniparser_getstring(), it return allocated + strings which the caller must free as necessary +- the API has been modifed such that all atalk_iniparser_get* funcs take a section and + a parameter as sepereta args instead of one string of the form "section:parameter" + in the original library + +CNID Database Daemons +===================== + +The CNID database daemons cnid_metad and cnid_dbd are a implementation of +the netatalk CNID database support that attempts to put all functionality +into separate daemons. +There is one cnid_dbd daemon per netatalk volume. The underlying database +structure is based on Berkeley DB and the database format is the same +as in the cdb CNID backend, so this can be used as a drop-in replacement. + +Advantages: + +- No locking issues or leftover locks due to crashed afpd daemons any + more. Since there is only one thread of control accessing the + database, no locking is needed and changes appear atomic. + +- Berkeley DB transactions are difficult to get right with several + processes attempting to access the CNID database simultanously. This + is much easier with a single process and the database can be made nearly + crashproof this way (at a performance cost). + +- No problems with user permissions and access to underlying database + files, the cnid_dbd process runs under a configurable user + ID that normally also owns the underlying database + and can be contacted by whatever afpd daemon accesses a volume. + +- If an afpd process crashes, the CNID database is unaffected. If the + process was making changes to the database at the time of the crash, + those changes will be rolled back entirely (transactions). + If the process was not using the database at the time of the crash, + no corrective action is necessary. In any case, database consistency + is assured. + +Disadvantages: + +- Performance in an environment of processes sharing the database + (files) is potentially better for two reasons: + + i) IPC overhead. + ii) r/o access to database pages is possible by more than one + process at once, r/w access is possible for nonoverlapping regions. + + The current implementation of cnid_dbd uses unix domain sockets as + the IPC mechanism. While this is not the fastest possible method, it + is very portable and the cnid_dbd IPC mechanisms can be extended to + use faster IPC (like mmap) on architectures where it is + supported. As a ballpark figure, 20000 requests/replies to the cnid_dbd + daemon take about 0.6 seconds on a Pentium III 733 Mhz running Linux + Kernel 2.4.18 using unix domain sockets. The requests are "empty" + (no database lookups/changes), so this is just the IPC + overhead. + + I have not measured the effects of the advantages of simultanous + database access. + + +Installation and configuration + +There are two executeables that will be built in etc/cnid_dbd and +installed into the systems binaries directories of netatalk +cnid_metad and cnid_dbd. cnid_metad should run all the +time with root permissions. It will be notified when an instance of +afpd starts up and will in turn make sure that a cnid_dbd daemon is +started for the volume that afpd wishes to access. The cnid_dbd daemon runs as +long as necessary and services any +other instances of afpd that access the volume. You can safely kill it +with SIGTERM, it will be restarted automatically by cnid_metad as soon +as the volume is accessed again. + +cnid_dbd changes to the Berkeley DB directory on startup and sets +effective UID and GID to owner and group of that directory. Database and +supporting files should therefore be writeable by that user/group. + +Current shortcomings: + +- The parameter file parsing of db_param is very simpleminded. It is +easy to cause buffer overruns and the like. +Also, there is no support for blanks (or weird characters) in +filenames for the usock_file parameter. + +- There is no protection against a malicious user connecting to the +cnid_dbd socket and changing the database. + +Documentation +============= +Netatalk documentation is in Docbook XML format. In order to build manpages +and the html documentation from the Docbook docs you need the following: + +1. Install `xsltproc` + +2. Get the latest Docbook XSL stylesheet distribution from: + https://sourceforge.net/project/showfiles.php?group_id=21935 + Tested docbook-xsl stylesheet version is 1.75.2. + +3. Fix indexterm bug in xsl stylesheet: + inside the xsl stylesheet distribution in manpages/inline.xsl remove these lines: + + + + + .\" + + + + + + + + + + : + + + +4. Add the following argument to configure + --with-docbook=PATH_TO_XML_STYLESHEET_DIR + +5. The manpages and html documentation are now automatically built when running `make html`. + +Editing Docbook Sources +----------------------- +Free WYSIWYG editor with only one minor drawback is XMLEditor from XMLmind: +http://www.xmlmind.com/xmleditor/persoedition.html + +Drawback: in order to be able to edit any of the nested xml files, you have to +"promote" them to valid Docbook files by referencing the Docbook DTD, insert as line 2+3: + + + +These changes will however prevent XMLeditor from opening the master xml file +manual.xml. Before further processing can be done these changes then have to be +reverted for any changed file.