14.4. Buses, Devices, and Drivers

So far, we have seen a great deal of low-level infrastructures and a relative shortage of examples. We try to make up for that in the rest of this chapter as we get into the higher levels of the Linux device model. To that end, we introduce a new virtual bus, which we call lddbus,^[1] and modify the scullp driver to "connect" to that bus.

^[1] The logical name for this bus, of course, would have been "sbus," but that name was already taken by a real, physical bus.

Once again, much of the material covered here will never be needed by many driver authors. Details at this level are generally handled at the bus level, and few authors need to add a new bus type. This information is useful, however, for anybody wondering what is happening inside the PCI, USB, etc. layers or who needs to make changes at that level.

14.4.1. Buses

A bus is a channel between the processor and one or more devices. For the purposes of the device model, all devices are connected via a bus, even if it is an internal, virtual, "platform" bus. Buses can plug into each other—a USB controller is usually a PCI device, for example. The device model represents the actual connections between buses and the devices they control.

In the Linux device model, a bus is represented by the bus_type structure, defined in <linux/device.h>. This structure looks like:

struct bus_type {
    char *name;
    struct subsystem subsys;
    struct kset drivers;
    struct kset devices;
    int (*match)(struct device *dev, struct device_driver *drv);
    struct device *(*add)(struct device * parent, char * bus_id);
    int (*hotplug) (struct device *dev, char **envp, 
                    int num_envp, char *buffer, int buffer_size);
    /* Some fields omitted */
};

The name field is the name of the bus, something such as pci. You can see from the structure that each bus is its own subsystem; these subsystems do not live at the top level in sysfs, however. Instead, they are found underneath the bus subsystem. A bus contains two ksets, representing the known drivers for that bus and all devices plugged into the bus. Then, there is a set of methods that we will get to shortly.

14.4.1.1 Bus registration

As we mentioned, the example source includes a virtual bus implementation called lddbus. This bus sets up its bus_type structure as follows:

struct bus_type ldd_bus_type = {
    .name = "ldd",
    .match = ldd_match,
    .hotplug  = ldd_hotplug,
};

Note that very few of the bus_type fields require initialization; most of that is handled by the device model core. We do have to specify the name of the bus, however, and any methods that go along with it.

Inevitably, a new bus must be registered with the system via a call to bus_register . The lddbus code does so in this way:

ret = bus_register(&ldd_bus_type);
if (ret)
    return ret;

This call can fail, of course, so the return value must always be checked. If it succeeds, the new bus subsystem has been added to the system; it is visible in sysfs under /sys/bus, and it is possible to start adding devices.

Should it be necessary to remove a bus from the system (when the associated module is removed, for example), bus_unregister should be called:

void bus_unregister(struct bus_type *bus);

14.4.1.2 Bus methods

There are several methods defined for the bus_type structure; they allow the bus code to serve as an intermediary between the device core and individual drivers. The methods defined in the 2.6.10 kernel are:

int (*match)(struct device *device, struct device_driver *driver);

This method is called, perhaps multiple times, whenever a new device or driver is added for this bus. It should return a nonzero value if the given device can be handled by the given driver. (We get to the details of the device and device_driver structures shortly). This function must be handled at the bus level, because that is where the proper logic exists; the core kernel cannot know how to match devices and drivers for every possible bus type.

int (*hotplug) (struct device *device, char **envp, int num_envp, char

*buffer, int buffer_size);

This method allows the bus to add variables to the environment prior to the generation of a hotplug event in user space. The parameters are the same as for the kset hotplug method (described in the earlier Section 14.3).

The lddbus driver has a very simple match function, which simply compares the driver and device names:

static int ldd_match(struct device *dev, struct device_driver *driver)
{
    return !strncmp(dev->bus_id, driver->name, strlen(driver->name));
}

When real hardware is involved, the match function usually makes some sort of comparison between the hardware ID provided by the device itself and the IDs supported by the driver.

The lddbus hotplug method looks like this:

static int ldd_hotplug(struct device *dev, char **envp, int num_envp,
        char *buffer, int buffer_size)
{
    envp[0] = buffer;
    if (snprintf(buffer, buffer_size, "LDDBUS_VERSION=%s",
                Version) >= buffer_size)
        return -ENOMEM;
    envp[1] = NULL;
    return 0;
}

Here, we add in the current revision number of the lddbus source, just in case anybody is curious.

14.4.1.3 Iterating over devices and drivers

If you are writing bus-level code, you may find yourself having to perform some operation on all devices or drivers that have been registered with your bus. It may be tempting to dig directly into the structures in the bus_type structure, but it is better to use the helper functions that have been provided.

To operate on every device known to the bus, use:

int bus_for_each_dev(struct bus_type *bus, struct device *start, 
                     void *data, int (*fn)(struct device *, void *));

This function iterates over every device on bus, passing the associated device structure to fn, along with the value passed in as data. If start is NULL, the iteration begins with the first device on the bus; otherwise iteration starts with the first device after start. If fn returns a nonzero value, iteration stops and that value is returned from bus_for_each_dev .

There is a similar function for iterating over drivers:

int bus_for_each_drv(struct bus_type *bus, struct device_driver *start, 
                     void *data, int (*fn)(struct device_driver *, void *));

This function works just like bus_for_each_dev, except, of course, that it works with drivers instead.

It should be noted that both of these functions hold the bus subsystem's reader/writer semaphore for the duration of the work. So an attempt to use the two of them together will deadlock—each will be trying to obtain the same semaphore. Operations that modify the bus (such as unregistering devices) will also lock up. So, use the bus_for_each functions with some care.

14.4.1.4 Bus attributes

Almost every layer in the Linux device model provides an interface for the addition of attributes, and the bus layer is no exception. The bus_attribute type is defined in <linux/device.h> as follows:

struct bus_attribute {
    struct attribute attr;
    ssize_t (*show)(struct bus_type *bus, char *buf);
    ssize_t (*store)(struct bus_type *bus, const char *buf, 
                     size_t count);
};

We have already seen struct attribute in Section 14.2.1. The bus_attribute type also includes two methods for displaying and setting the value of the attribute. Most device model layers above the kobject level work this way.

A convenience macro has been provided for the compile-time creation and initialization of bus_attribute structures:

BUS_ATTR(name, mode, show, store);

This macro declares a structure, generating its name by prepending the string bus_attr_ to the given name.

Any attributes belonging to a bus should be created explicitly with bus_create_file:

int bus_create_file(struct bus_type *bus, struct bus_attribute *attr);

Attributes can also be removed with:

void bus_remove_file(struct bus_type *bus, struct bus_attribute *attr);

The lddbus driver creates a simple attribute file containing, once again, the source version number. The show method and bus_attribute structure are set up as follows:

static ssize_t show_bus_version(struct bus_type *bus, char *buf)
{
    return snprintf(buf, PAGE_SIZE, "%s\n", Version);
}

static BUS_ATTR(version, S_IRUGO, show_bus_version, NULL);

Creating the attribute file is done at module load time:

if (bus_create_file(&ldd_bus_type, &bus_attr_version))
    printk(KERN_NOTICE "Unable to create version attribute\n");

This call creates an attribute file (/sys/bus/ldd/version) containing the revision number for the lddbus code.

14.4.2. Devices

At the lowest level, every device in a Linux system is represented by an instance of struct device:

struct device {
    struct device *parent;
    struct kobject kobj;
    char bus_id[BUS_ID_SIZE];
    struct bus_type *bus;
    struct device_driver *driver;
    void *driver_data;
    void (*release)(struct device *dev);
    /* Several fields omitted */
};

There are many other struct device fields that are of interest only to the device core code. These fields, however, are worth knowing about:

struct device *parent

The device's "parent" device—the device to which it is attached. In most cases, a parent device is some sort of bus or host controller. If parent is NULL, the device is a top-level device, which is not usually what you want.

struct kobject kobj;

The kobject that represents this device and links it into the hierarchy. Note that, as a general rule, device->kobj->parent is equal to &device->parent->kobj.

char bus_id[BUS_ID_SIZE];

A string that uniquely identifies this device on the bus. PCI devices, for example, use the standard PCI ID format containing the domain, bus, device, and function numbers.

struct bus_type *bus;

Identifies which kind of bus the device sits on.

struct device_driver *driver;

The driver that manages this device; we examine struct device_driver in the next section.

void *driver_data;

A private data field that may be used by the device driver.

void (*release)(struct device *dev);

The method is called when the last reference to the device is removed; it is called from the embedded kobject's release method. All device structures registered with the core must have a release method, or the kernel prints out scary complaints.

At a minimum, the parent, bus_id, bus, and release fields must be set before the device structure can be registered.

14.4.2.1 Device registration

The usual set of registration and unregistration functions exists:

int device_register(struct device *dev);
void device_unregister(struct device *dev);

We have seen how the lddbus code registers its bus type. However, an actual bus is a device and must be registered separately. For simplicity, the lddbus module supports only a single virtual bus, so the driver sets up its device at compile time:

static void ldd_bus_release(struct device *dev)
{
    printk(KERN_DEBUG "lddbus release\n");
}
    
struct device ldd_bus = {
    .bus_id   = "ldd0",
    .release  = ldd_bus_release
};

This is a top-level bus, so the parent and bus fields are left NULL. We have a simple, no-op release method, and, as the first (and only) bus, its name is ldd0. This bus device is registered with:

ret = device_register(&ldd_bus);
if (ret)
    printk(KERN_NOTICE "Unable to register ldd0\n");

Once that call is complete, the new bus can be seen under /sys/devices in sysfs. Any devices added to this bus then shows up under /sys/devices/ldd0/.

14.4.2.2 Device attributes

Device entries in sysfs can have attributes. The relevant structure is:

struct device_attribute {
    struct attribute attr;
    ssize_t (*show)(struct device *dev, char *buf);
    ssize_t (*store)(struct device *dev, const char *buf, 
                     size_t count);
};

These attribute structures can be set up at compile time with this macro:

DEVICE_ATTR(name, mode, show, store);

The resulting structure is named by prepending dev_attr_ to the given name. The actual management of attribute files is handled with the usual pair of functions:

int device_create_file(struct device *device, 
                       struct device_attribute *entry);
void device_remove_file(struct device *dev, 
                        struct device_attribute *attr);

The dev_attrs field of struct bus_type points to a list of default attributes created for every device added to that bus.

14.4.2.3 Device structure embedding

The device structure contains the information that the device model core needs to model the system. Most subsystems, however, track additional information about the devices they host. As a result, it is rare for devices to be represented by bare device structures; instead, that structure, like kobject structures, is usually embedded within a higher-level representation of the device. If you look at the definitions of struct pci_dev or struct usb_device, you will find a struct device buried inside. Usually, low-level drivers are not even aware of that struct device, but there can be exceptions.

The lddbus driver creates its own device type (struct ldd_device) and expects individual device drivers to register their devices using that type. It is a simple structure:

struct ldd_device {
    char *name;
    struct ldd_driver *driver;
    struct device dev;
};

#define to_ldd_device(dev) container_of(dev, struct ldd_device, dev);

This structure allows the driver to provide an actual name for the device (which can be distinct from its bus ID, stored in the device structure) and a pointer to driver information. Structures for real devices usually also contain information about the vendor, device model, device configuration, resources used, and so on. Good examples can be found in struct pci_dev (<linux/pci.h>) or struct usb_device (<linux/usb.h>). A convenience macro (to_ldd_device) is also defined for struct ldd_device to make it easy to turn pointers to the embedded device structure into ldd_device pointers.

The registration interface exported by lddbus looks like this:

int register_ldd_device(struct ldd_device *ldddev)
{
    ldddev->dev.bus = &ldd_bus_type;
    ldddev->dev.parent = &ldd_bus;
    ldddev->dev.release = ldd_dev_release;
    strncpy(ldddev->dev.bus_id, ldddev->name, BUS_ID_SIZE);
    return device_register(&ldddev->dev);
}
EXPORT_SYMBOL(register_ldd_device);

Here, we simply fill in some of the embedded device structure fields (which individual drivers should not need to know about), and register the device with the driver core. If we wanted to add bus-specific attributes to the device, we could do so here.

To show how this interface is used, let us introduce another sample driver, which we have called sculld. It is yet another variant on the scullp driver first introduced in Chapter 8. It implements the usual memory area device, but sculld also works with the Linux device model by way of the lddbus interface.

The sculld driver adds an attribute of its own to its device entry; this attribute, called dev, simply contains the associated device number. This attribute could be used by a module loading the script or the hotplug subsystem to automatically create device nodes when the device is added to the system. The setup for this attribute follows the usual patterns:

static ssize_t sculld_show_dev(struct device *ddev, char *buf)
{
    struct sculld_dev *dev = ddev->driver_data;

    return print_dev_t(buf, dev->cdev.dev);
}

static DEVICE_ATTR(dev, S_IRUGO, sculld_show_dev, NULL);

Then, at initialization time, the device is registered, and the dev attribute is created through the following function:

static void sculld_register_dev(struct sculld_dev *dev, int index)
{
    sprintf(dev->devname, "sculld%d", index);
    dev->ldev.name = dev->devname;
    dev->ldev.driver = &sculld_driver;
    dev->ldev.dev.driver_data = dev;
    register_ldd_device(&dev->ldev);
    device_create_file(&dev->ldev.dev, &dev_attr_dev);
}

Note that we make use of the driver_data field to store the pointer to our own, internal device structure.

14.4.3. Device Drivers

The device model tracks all of the drivers known to the system. The main reason for this tracking is to enable the driver core to match up drivers with new devices. Once drivers are known objects within the system, however, a number of other things become possible. Device drivers can export information and configuration variables that are independent of any specific device, for example.

Drivers are defined by the following structure:

struct device_driver {
    char *name;
    struct bus_type *bus;
    struct kobject kobj;
    struct list_head devices;
    int (*probe)(struct device *dev);
    int (*remove)(struct device *dev);
    void (*shutdown) (struct device *dev);
};

Once again, several of the structure's fields have been omitted (see <linux/device.h> for the full story). Here, name is the name of the driver (it shows up in sysfs), bus is the type of bus this driver works with, kobj is the inevitable kobject, devices is a list of all devices currently bound to this driver, probe is a function called to query the existence of a specific device (and whether this driver can work with it), remove is called when the device is removed from the system, and shutdown is called at shutdown time to quiesce the device.

The form of the functions for working with device_driver structures should be looking familiar by now (so we cover them very quickly). The registration functions are:

int driver_register(struct device_driver *drv);
void driver_unregister(struct device_driver *drv);

The usual attribute structure exists:

struct driver_attribute {
    struct attribute attr;
    ssize_t (*show)(struct device_driver *drv, char *buf);
    ssize_t (*store)(struct device_driver *drv, const char *buf, 
                     size_t count);
};
DRIVER_ATTR(name, mode, show, store);

And attribute files are created in the usual way:

int driver_create_file(struct device_driver *drv, 
                       struct driver_attribute *attr);
void driver_remove_file(struct device_driver *drv, 
                        struct driver_attribute *attr);

The bus_type structure contains a field (drv_attrs) that points to a set of default attributes, which are created for all drivers associated with that bus.

14.4.3.1 Driver structure embedding

As is the case with mos t driver core structures, the device_driver structure is usually found embedded within a higher-level, bus-specific structure. The lddbus subsystem would never go against such a trend, so it has defined its own ldd_driver structure:

struct ldd_driver {
    char *version;
    struct module *module;
    struct device_driver driver;
    struct driver_attribute version_attr;
};

#define to_ldd_driver(drv) container_of(drv, struct ldd_driver, driver);

Here, we require each driver to provide its current software version, and lddbus exports that version string for every driver it knows about. The bus-specific driver registration function is:

int register_ldd_driver(struct ldd_driver *driver)
{
    int ret;
    
    driver->driver.bus = &ldd_bus_type;
    ret = driver_register(&driver->driver);
    if (ret)
        return ret;
    driver->version_attr.attr.name = "version";
    driver->version_attr.attr.owner = driver->module;
    driver->version_attr.attr.mode = S_IRUGO;
    driver->version_attr.show = show_version;
    driver->version_attr.store = NULL;
    return driver_create_file(&driver->driver, &driver->version_attr);
}

The first half of the function simply registers the low-level device_driver structure with the core; the rest sets up the version attribute. Since this attribute is created at runtime, we can't use the DRIVER_ATTR macro; instead, the driver_attribute structure must be filled in by hand. Note that we set the owner of the attribute to the driver module, rather than the lddbus module; the reason for this can be seen in the implementation of the show function for this attribute:

static ssize_t show_version(struct device_driver *driver, char *buf)
{
    struct ldd_driver *ldriver = to_ldd_driver(driver);

    sprintf(buf, "%s\n", ldriver->version);
    return strlen(buf);
}

One might think that the attribute owner should be the lddbus module, since the function that implements the attribute is defined there. This function, however, is working with the ldd_driver structure created (and owned) by the driver itself. If that structure were to go away while a user-space process tried to read the version number, things could get messy. Designating the driver module as the owner of the attribute prevents the module from being unloaded, while user-space holds the attribute file open. Since each driver module creates a reference to the lddbus module, we can be sure that lddbus will not be unloaded at an inopportune time.

For completeness, sculld creates its ldd_driver structure as follows:

static struct ldd_driver sculld_driver = {
    .version = "$Revision: 1.1 $",
    .module = THIS_MODULE,
    .driver = {
        .name = "sculld",
    },
};

A simple call to register_ldd_driver adds it to the system. Once initialization is complete, the driver information can be seen in sysfs:

$ tree /sys/bus/ldd/drivers
/sys/bus/ldd/drivers
`-- sculld
    |-- sculld0 -> ../../../../devices/ldd0/sculld0
    |-- sculld1 -> ../../../../devices/ldd0/sculld1
    |-- sculld2 -> ../../../../devices/ldd0/sculld2
    |-- sculld3 -> ../../../../devices/ldd0/sculld3
    `-- version