FMC: add documentation for the core

This is selected sections of the current manual for fmc-bus, as
developed outside of the kernel before submission.

Like the other patches in this set, it corresponds to commit ab23167f of
the repository at ohwr.org
Signed-off-by: Alessandro Rubini <rubini@gnudd.com>
Acked-by: Juan David Gonzalez Cobas <dcobas@cern.ch>
Acked-by: Emilio G. Cota <cota@braap.org>
Acked-by: Samuel Iglesias Gonsalvez <siglesias@igalia.com>
Acked-by: Rob Landley <rob@landley.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Documentation/00-INDEX

@@ -187,6 +187,8 @@ firmware_class/
 	- request_firmware() hotplug interface info.
 flexible-arrays.txt
 	- how to make use of flexible sized arrays in linux
+fmc/
+	- information about the FMC bus abstraction
 frv/
 	- Fujitsu FR-V Linux documentation.
 futex-requeue-pi.txt

Documentation/fmc/00-INDEX

+
+Documentation in this directory comes from sections of the manual we
+wrote for the externally-developed fmc-bus package. The complete
+manual as of today (2013-02) is available in PDF format at
+http://www.ohwr.org/projects/fmc-bus/files
+
+00-INDEX
+	- this file.
+
+FMC-and-SDB.txt
+	- What are FMC and SDB, basic concepts for this framework
+
+API.txt
+	- The functions that are exported by the bus driver
+
+parameters.txt
+	- The module parameters
+
+carrier.txt
+	- writing a carrier (a device)
+
+mezzanine.txt
+	- writing code for your mezzanine (a driver)
+
+identifiers.txt
+	- how identification and matching works

Documentation/fmc/API.txt

+Functions Exported by fmc.ko
+****************************
+
+The FMC core exports the usual 4 functions that are needed for a bus to
+work, and a few more:
+
+        int fmc_driver_register(struct fmc_driver *drv);
+        void fmc_driver_unregister(struct fmc_driver *drv);
+        int fmc_device_register(struct fmc_device *fmc);
+        void fmc_device_unregister(struct fmc_device *fmc);
+
+        int fmc_device_register_n(struct fmc_device **fmc, int n);
+        void fmc_device_unregister_n(struct fmc_device **fmc, int n);
+
+        uint32_t fmc_readl(struct fmc_device *fmc, int offset);
+        void fmc_writel(struct fmc_device *fmc, uint32_t val, int off);
+        void *fmc_get_drvdata(struct fmc_device *fmc);
+        void fmc_set_drvdata(struct fmc_device *fmc, void *data);
+
+        int fmc_reprogram(struct fmc_device *f, struct fmc_driver *d, char *gw,
+                          int sdb_entry);
+
+The data structure that describe a device is detailed in *note FMC
+Device::, the one that describes a driver is detailed in *note FMC
+Driver::.  Please note that structures of type fmc_device must be
+allocated by the caller, but must not be released after unregistering.
+The fmc-bus itself takes care of releasing the structure when their use
+count reaches zero - actually, the device model does that in lieu of us.
+
+The functions to register and unregister n devices are meant to be used
+by carriers that host more than one mezzanine. The devices must all be
+registered at the same time because if the FPGA is reprogrammed, all
+devices in the array are affected. Usually, the driver matching the
+first device will reprogram the FPGA, so other devices must know they
+are already driven by a reprogrammed FPGA.
+
+If a carrier hosts slots that are driven by different FPGA devices, it
+should register as a group only mezzanines that are driven by the same
+FPGA, for the reason outlined above.
+
+Finally, the fmc_reprogram function calls the reprogram method (see
+*note The API Offered by Carriers:: and also scans the memory area for
+an SDB tree. You can pass -1 as sdb_entry to disable such scan.
+Otherwise, the function fails if no tree is found at the specified
+entry point.  The function is meant to factorize common code, and by
+the time you read this it is already used by the spec-sw and fine-delay
+modules.

Documentation/fmc/FMC-and-SDB.txt

+
+FMC (FPGA Mezzanine Card) is the standard we use for our I/O devices,
+in the context of White Rabbit and related hardware.
+
+In our I/O environments we need to write drivers for each mezzanine
+card, and such drivers must work regardless of the carrier being used.
+To achieve this, we abstract the FMC interface.
+
+We have a carrier for PCI-E called SPEC and one for VME called SVEC,
+but more are planned.  Also, we support stand-alone devices (usually
+plugged on a SPEC card), controlled through Etherbone, developed by GSI.
+
+Code and documentation for the FMC bus was born as part of the spec-sw
+project, but now it lives in its own project. Other projects, i.e.
+software support for the various carriers, should include this as a
+submodule.
+
+The most up to date version of code and documentation is always
+available from the repository you can clone from:
+
+        git://ohwr.org/fmc-projects/fmc-bus.git (read-only)
+        git@ohwr.org:fmc-projects/fmc-bus.git (read-write for developers)
+
+Selected versions of the documentation, as well as complete tar
+archives for selected revisions are placed to the Files section of the
+project: `http://www.ohwr.org/projects/fmc-bus/files'
+
+
+What is FMC
+***********
+
+FMC, as said, stands for "FPGA Mezzanine Card". It is a standard
+developed by the VME consortium called VITA (VMEbus International Trade
+Association and ratified by ANSI, the American National Standard
+Institute.  The official documentation is called "ANSI-VITA 57.1".
+
+The FMC card is an almost square PCB, around 70x75 millimeters, that is
+called mezzanine in this document.  It usually lives plugged into
+another PCB for power supply and control; such bigger circuit board is
+called carrier from now on, and a single carrier may host more than one
+mezzanine.
+
+In the typical application the mezzanine is mostly analog while the
+carrier is mostly digital, and hosts an FPGA that must be configured to
+match the specific mezzanine and the desired application. Thus, you may
+need to load different FPGA images to drive different instances of the
+same mezzanine.
+
+FMC, as such, is not a bus in the usual meaning of the term, because
+most carriers have only one connector, and carriers with several
+connectors have completely separate electrical connections to them.
+This package, however, implements a bus as a software abstraction.
+
+
+What is SDB
+***********
+
+SDB (Self Describing Bus) is a set of data structures that we use for
+enumerating the internal structure of an FPGA image. We also use it as
+a filesystem inside the FMC EEPROM.
+
+SDB is not mandatory for use of this FMC kernel bus, but if you have SDB
+this package can make good use of it.  SDB itself is developed in the
+fpga-config-space OHWR project. The link to the repository is
+`git://ohwr.org/hdl-core-lib/fpga-config-space.git' and what is used in
+this project lives in the sdbfs subdirectory in there.
+
+SDB support for FMC is described in *note FMC Identification:: and
+*note SDB Support::
+
+
+SDB Support
+***********
+
+The fmc.ko bus driver exports a few functions to help drivers taking
+advantage of the SDB information that may be present in your own FPGA
+memory image.
+
+The module exports the following functions, in the special header
+<linux/fmc-sdb.h>. The linux/ prefix in the name is there because we
+plan to submit it upstream in the future, and don't want to force
+changes on our drivers if that happens.
+
+         int fmc_scan_sdb_tree(struct fmc_device *fmc, unsigned long address);
+         void fmc_show_sdb_tree(struct fmc_device *fmc);
+         signed long fmc_find_sdb_device(struct sdb_array *tree, uint64_t vendor,
+                                         uint32_t device, unsigned long *sz);
+         int fmc_free_sdb_tree(struct fmc_device *fmc);

Documentation/fmc/identifiers.txt

+FMC Identification
+******************
+
+The FMC standard requires every compliant mezzanine to carry
+identification information in an I2C EEPROM.  The information must be
+laid out according to the "IPMI Platform Management FRU Information",
+where IPMI is a lie I'd better not expand, and FRU means "Field
+Replaceable Unit".
+
+The FRU information is an intricate unreadable binary blob that must
+live at offset 0 of the EEPROM, and typically extends for a few hundred
+bytes. The standard allows the application to use all the remaining
+storage area of the EEPROM as it wants.
+
+This chapter explains how to create your own EEPROM image and how to
+write it in your mezzanine, as well as how devices and drivers are
+paired at run time.  EEPROM programming uses tools that are part of this
+package and SDB (part of the fpga-config-space package).
+
+The first sections are only interesting for manufacturers who need to
+write the EEPROM. If you are just a software developer writing an FMC
+device or driver, you may jump straight to *note SDB Support::.
+
+
+Building the FRU Structure
+==========================
+
+If you want to know the internals of the FRU structure and despair, you
+can retrieve the document from
+`http://download.intel.com/design/servers/ipmi/FRU1011.pdf' .  The
+standard is awful and difficult without reason, so we only support the
+minimum mandatory subset - we create a simple structure and parse it
+back at run time, but we are not able to either generate or parse more
+arcane features like non-english languages and 6-bit text.  If you need
+more items of the FRU standard for your boards, please submit patches.
+
+This package includes the Python script that Matthieu Cattin wrote to
+generate the FRU binary blob, based on an helper libipmi by Manohar
+Vanga and Matthieu himself.  I changed the test script to receive
+parameters from the command line or from the environment (the command
+line takes precedence)
+
+To make a long story short, in order to build a standard-compliant
+binary file to be burned in your EEPROM, you need the following items:
+
+        Environment    Opt     Official Name          Default
+---------------------------------------------------------------------
+        FRU_VENDOR     -v      "Board Manufacturer"   fmc-example
+        FRU_NAME       -n      "Board Product Name"   mezzanine
+        FRU_SERIAL     -s      `Board Serial Number"  0001
+        FRU_PART       -p      "Board Part Number"    sample-part
+        FRU_OUTPUT     -o      not applicable         /dev/stdout
+
+The "Official Name" above is what you find in the FRU official
+documentation, chapter 11, page 7 ("Board Info Area Format").  The
+output option is used to save the generated binary to a specific file
+name instead of stdout.
+
+You can pass the items to the FRU generator either in the environment
+or on the command line.  This package has currently no support for
+specifying power consumption or such stuff, but I plan to add it as
+soon as I find some time for that.
+
+FIXME: consumption etc for FRU are here or in PTS?
+
+The following example creates a binary image for a specific board:
+
+        ./tools/fru-generator -v CERN -n FmcAdc100m14b4cha \
+               -s HCCFFIA___-CR000003 -p EDA-02063-V5-0 > eeprom.bin
+
+The following example shows a script that builds several binary EEPROM
+images for a series of boards, changing the serial number for each of
+them. The script uses a mix of environment variables and command line
+options, and uses the same string patterns shown above.
+
+        #!/bin/sh
+
+        export FRU_VENDOR="CERN"
+        export FRU_NAME="FmcAdc100m14b4cha"
+        export FRU_PART="EDA-02063-V5-0"
+
+        serial="HCCFFIA___-CR"
+
+        for number in $(seq 1 50); do
+           # build number-string "ns"
+           ns="$(printf %06d $number)"
+           ./fru-generator -s "${serial}${ns}" > eeprom-${ns}.bin
+        done
+
+
+Using SDB-FS in the EEPROM
+==========================
+
+If you want to use SDB as a filesystem in the EEPROM device within the
+mezzanine, you should create one such filesystem using gensdbfs, from
+the fpga-config-space package on OHWR.
+
+By using an SBD filesystem you can cluster several files in a single
+EEPROM, so both the host system and a soft-core running in the FPGA (if
+any) can access extra production-time information.
+
+We chose to use SDB as a storage filesystem because the format is very
+simple, and both the host system and the soft-core will likely already
+include support code for such format. The SDB library offered by the
+fpga-config-space is less than 1kB under LM32, so it proves quite up to
+the task.
+
+The SDB entry point (which acts as a directory listing) cannot live at
+offset zero in the flash device, because the FRU information must live
+there.  To avoid wasting precious storage space while still allowing
+for more-than-minimal FRU structures, the fmc.ko will look for the SDB
+record at address 256, 512 and 1024.
+
+In order to generate the complete EEPROM image you'll need a
+configuration file for gensdbfs: you tell the program where to place
+the sdb entry point, and you must force the FRU data file to be placed
+at the beginning of the storage device. If needed, you can also place
+other files at a special offset (we sometimes do it for backward
+compatibility with drivers we wrote before implementing SDB for flash
+memory).
+
+The directory tools/sdbfs of this package includes a well-commented
+example that you may want to use as a starting point (the comments are
+in the file called -SDB-CONFIG-).  Reading documentation for gensdbfs
+is a suggested first step anyways.
+
+This package (generic FMC bus support) only accesses two files in the
+EEPROM: the FRU information, at offset zero, with a suggested filename
+of IPMI-FRU and the short name for the mezzanine, in a file called
+name. The IPMI-FRU name is not mandatory, but a strongly suggested
+choice; the name filename is mandatory, because this is the preferred
+short name used by the FMC core.  For example, a name of "fdelay" may
+supplement a Product Name like "FmcDelay1ns4cha" - exactly as
+demonstrated in `tools/sdbfs'.
+
+Note: SDB access to flash memory is not yet supported, so the short
+name currently in use is just the "Product Name" FRU string.
+
+The example in tools/sdbfs includes an extra file, that is needed by
+the fine-delay driver, and must live at a known address of 0x1800.  By
+running gensdbfs on that directory you can output your binary EEPROM
+image (here below spusa$ is the shell prompt):
+
+        spusa$ ../fru-generator -v CERN -n FmcDelay1ns4cha -s proto-0 \
+                      -p EDA-02267-V3 > IPMI-FRU
+        spusa$ ls -l
+        total 16
+        -rw-rw-r-- 1 rubini staff 975 Nov 19 18:08 --SDB-CONFIG--
+        -rw-rw-r-- 1 rubini staff 216 Nov 19 18:13 IPMI-FRU
+        -rw-rw-r-- 1 rubini staff  11 Nov 19 18:04 fd-calib
+        -rw-rw-r-- 1 rubini staff   7 Nov 19 18:04 name
+        spusa$ sudo gensdbfs . /lib/firmware/fdelay-eeprom.bin
+        spusa$ sdb-read -l -e 0x100 /lib/firmware/fdelay-eeprom.bin
+        /home/rubini/wip/sdbfs/userspace/sdb-read: listing format is to be defined
+        46696c6544617461:2e202020  00000100-000018ff .
+        46696c6544617461:6e616d65  00000200-00000206 name
+        46696c6544617461:66642d63  00001800-000018ff fd-calib
+        46696c6544617461:49504d49  00000000-000000d7 IPMI-FRU
+        spusa$ ../fru-dump /lib/firmware/fdelay-eeprom.bin
+        /lib/firmware/fdelay-eeprom.bin: manufacturer: CERN
+        /lib/firmware/fdelay-eeprom.bin: product-name: FmcDelay1ns4cha
+        /lib/firmware/fdelay-eeprom.bin: serial-number: proto-0
+        /lib/firmware/fdelay-eeprom.bin: part-number: EDA-02267-V3
+
+As expected, the output file is both a proper sdbfs object and an IPMI
+FRU information blob. The fd-calib file lives at offset 0x1800 and is
+over-allocated to 256 bytes, according to the configuration file for
+gensdbfs.

Documentation/fmc/mezzanine.txt

+FMC Driver
+**********
+
+An FMC driver is concerned with the specific mezzanine and associated
+gateware. As such, it is expected to be independent of the carrier
+being used: it will perform I/O accesses only by means of
+carrier-provided functions.
+
+The matching between device and driver is based on the content of the
+EEPROM (as mandated by the FMC standard) or by the actual cores
+configured in the FPGA; the latter technique is used when the FPGA is
+already programmed when the device is registered to the bus core.
+
+In some special cases it is possible for a driver to directly access
+FPGA registers, by means of the `fpga_base' field of the device
+structure. This may be needed for high-bandwidth peripherals like fast
+ADC cards. If the device module registered a remote device (for example
+by means of Etherbone), the `fpga_base' pointer will be NULL.
+Therefore, drivers must be ready to deal with NULL base pointers, and
+fail gracefully.  Most driver, however, are not expected to access the
+pointer directly but run fmc_readl and fmc_writel instead, which will
+work in any case.
+
+In even more special cases, the driver may access carrier-specific
+functionality: the `carrier_name' string allows the driver to check
+which is the current carrier and make use of the `carrier_data'
+pointer.  We chose to use carrier names rather than numeric identifiers
+for greater flexibility, but also to avoid a central registry within
+the `fmc.h' file - we hope other users will exploit our framework with
+their own carriers.  An example use of carrier names is in GPIO setup
+(see *note The GPIO Abstraction::), although the name match is not
+expected to be performed by the driver.  If you depend on specific
+carriers, please check the carrier name and fail gracefully if your
+driver finds it is running in a yet-unknown-to-it environment.
+
+
+ID Table
+========
+
+Like most other Linux drivers, and FMC driver must list all the devices
+which it is able to drive.  This is usually done by means of a device
+table, but in FMC we can match hardware based either on the contents of
+their EEPROM or on the actual FPGA cores that can be enumerated.
+Therefore, we have two tables of identifiers.
+
+Matching of FRU information depends on two names, the manufacturer (or
+vendor) and the device (see *note FMC Identification::); for
+flexibility during production (i.e. before writing to the EEPROM) the
+bus supports a catch-all driver that specifies NULL strings. For this
+reason, the table is specified as pointer-and-length, not a a
+null-terminated array - the entry with NULL names can be a valid entry.
+
+Matching on FPGA cores depends on two numeric fields: the 64-bit vendor
+number and the 32-bit device number. Support for matching based on
+class is not yet implemented.  Each device is expected to be uniquely
+identified by an array of cores (it matches if all of the cores are
+instantiated), and for consistency the list is passed as
+pointer-and-length.  Several similar devices can be driven by the same
+driver, and thus the driver specifies and array of such arrays.
+
+The complete set of involved data structures is thus the following:
+
+        struct fmc_fru_id { char *manufacturer; char *product_name; };
+        struct fmc_sdb_one_id { uint64_t vendor; uint32_t device; };
+        struct fmc_sdb_id { struct fmc_sdb_one_id *cores; int cores_nr; };
+
+        struct fmc_device_id {
+                struct fmc_fru_id *fru_id; int fru_id_nr;
+                struct fmc_sdb_id *sdb_id; int sdb_id_nr;
+        };
+
+A better reference, with full explanation, is the <linux/fmc.h> header.
+
+
+Module Parameters
+=================
+
+Most of the FMC drivers need the same set of kernel parameters. This
+package includes support to implement common parameters by means of
+fields in the `fmc_driver' structure and simple macro definitions.
+
+The parameters are carrier-specific, in that they rely on the busid
+concept, that varies among carriers. For the SPEC, the identifier is a
+PCI bus and devfn number, 16 bits wide in total; drivers for other
+carriers will most likely offer something similar but not identical,
+and some code duplication is unavoidable.
+
+This is the list of parameters that are common to several modules to
+see how they are actually used, please look at spec-trivial.c.
+
+`busid='
+     This is an array of integers, listing carrier-specific
+     identification numbers. For PIC, for example, `0x0400' represents
+     bus 4, slot 0.  If any such ID is specified, the driver will only
+     accept to drive cards that appear in the list (even if the FMC ID
+     matches). This is accomplished by the validate carrier method.
+
+`gateware='
+     The argument is an array of strings. If no busid= is specified,
+     the first string of gateware= is used for all cards; otherwise the
+     identifiers and gateware names are paired one by one, in the order
+     specified.
+
+`show_sdb='
+     For modules supporting it, this parameter asks to show the SDB
+     internal structure by means of kernel messages. It is disabled by
+     default because those lines tend to hide more important messages,
+     if you look at the system console while loading the drivers.
+     Note: the parameter is being obsoleted, because fmc.ko itself now
+     supports dump_sdb= that applies to every client driver.
+
+
+For example, if you are using the trivial driver to load two different
+gateware files to two different cards, you can use the following
+parameters to load different binaries to the cards, after looking up
+the PCI identifiers. This has been tested with a SPEC carrier.
+
+        insmod fmc-trivial.ko \
+                              busid=0x0200,0x0400 \
+                              gateware=fmc/fine-delay.bin,fmc/simple-dio.bin
+
+Please note that not all sub-modules support all of those parameters.
+You can use modinfo to check what is supported by each module.

Documentation/fmc/parameters.txt

+Module Parameters in fmc.ko
+***************************
+
+The core driver receives two module parameters, meant to help debugging
+client modules. Both parameters can be modified by writing to
+/sys/module/fmc/parameters/, because they are used when client drivers
+are devices are registered, not when fmc.ko is loaded.
+
+`dump_eeprom='
+     If not zero, the parameter asks the bus controller to dump the
+     EEPROM of any device that is registered, using printk.
+
+`dump_sdb='
+     If not zero, the parameter prints the SDB tree of every FPGA it is
+     loaded by fmc_reprogram(). If greater than one, it asks to dump
+     the binary content of SDB records.  This currently only dumps the
+     top-level SDB array, though.
+
+
+EEPROM dumping avoids repeating lines, since most of the contents is
+usually empty and all bits are one or zero. This is an example of the
+output:
+
+        [ 6625.850480] spec 0000:02:00.0: FPGA programming successful
+        [ 6626.139949] spec 0000:02:00.0: Manufacturer: CERN
+        [ 6626.144666] spec 0000:02:00.0: Product name: FmcDelay1ns4cha
+        [ 6626.150370] FMC: mezzanine 0: 0000:02:00.0 on SPEC
+        [ 6626.155179] FMC: dumping eeprom 0x2000 (8192) bytes
+        [ 6626.160087] 0000: 01 00 00 01  00 0b 00 f3  01 0a 00 a5  85 87 c4 43
+        [ 6626.167069] 0010: 45 52 4e cf  46 6d 63 44  65 6c 61 79  31 6e 73 34
+        [ 6626.174019] 0020: 63 68 61 c7  70 72 6f 74  6f 2d 30 cc  45 44 41 2d
+        [ 6626.180975] 0030: 30 32 32 36  37 2d 56 33  da 32 30 31  32 2d 31 31
+        [...]
+        [ 6626.371366] 0200: 66 64 65 6c  61 79 0a 00  00 00 00 00  00 00 00 00
+        [ 6626.378359] 0210: 00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00
+        [ 6626.385361] [...]
+        [ 6626.387308] 1800: 70 6c 61 63  65 68 6f 6c  64 65 72 ff  ff ff ff ff
+        [ 6626.394259] 1810: ff ff ff ff  ff ff ff ff  ff ff ff ff  ff ff ff ff
+        [ 6626.401250] [...]
+
+The dump of SDB looks like the following; the example shows the simple
+golden gateware for the SPEC card, removing the leading timestamps to
+fit the page:
+
+        spec 0000:02:00.0: SDB: 00000651:e6a542c9 WB4-Crossbar-GSI
+        spec 0000:02:00.0: SDB: 0000ce42:ff07fc47 WR-Periph-Syscon (00000000-000000ff)
+        FMC: mezzanine 0: 0000:02:00.0 on SPEC
+        FMC: poor dump of sdb first level:
+        0000: 53 44 42 2d  00 02 01 00  00 00 00 00  00 00 00 00
+        0010: 00 00 00 00  00 00 01 ff  00 00 00 00  00 00 06 51
+        0020: e6 a5 42 c9  00 00 00 02  20 12 05 11  57 42 34 2d
+        0030: 43 72 6f 73  73 62 61 72  2d 47 53 49  20 20 20 00
+        0040: 00 00 01 01  00 00 00 07  00 00 00 00  00 00 00 00
+        0050: 00 00 00 00  00 00 00 ff  00 00 00 00  00 00 ce 42
+        0060: ff 07 fc 47  00 00 00 01  20 12 03 05  57 52 2d 50
+        0070: 65 72 69 70  68 2d 53 79  73 63 6f 6e  20 20 20 01