Less known Solaris Features: iSCSI with COMSTAR
A while ago, i wrote a tutorial about using iSCSI in Opensolaris and Solaris 10. While the tutorial is still valid for Solaris 10, Opensolaris got a new, more efficient iSCSI Target. iSCSI in Opensolaris is a part of a more generic in-kernel framework to provide SCSI Target services. This framework isn’t just capable to deliver iSCSI, you can have FCoE, SRP, FC targets as well. This generic approach led to a different administrative model. Thus when you want to configure iSCSI on a OpenSolaris system and you want the new framework (the old userland based version is still available) you have to configure the target side differently. This tutorial will do pretty much the same than the old iSCSI tutorial, just with the new framework to give you an overview about the changes.
Why does COMSTAR need a different administrative model?
The reason for the this change is somewhat based in the point for it’s name: It’s a COmmon Multiprotocol Scsi TARget framework. iSCSI, SRP, iSER, SAS, FCoE, FC have something in common. They use SCSI commands. SCSI isn’t just a physical interface. It’s a way to transmit data over an interface. It’s a set of commands. And independently from the physical interface technology you use, the commands are pretty much the same. A good sign for this commonality is the same jargon in all this technologies: The entity executing SCSI command is called target in Fibrechannel jargon as well in iSCSI as well in SCSI over RDMA over Infiniband. The same is valid for the entity issuing commands to a target. It’s called initiator in all those protocols as well.
The differences between all the proctols mentioned above is the way, how the SCSI commands are transmitted over the wire. This led to an obvious idea: Separating the layer providing the SCSI target from the layer providing the protcol to access the SCSI target. The component providing this protocols are called “Port Provider” in COMSTAR.
On the other side, COMSTAR isn’t just meant to provide access to SCSI disks. Providing tape devices is in the focus as well. The component providing disk or tape devices is called “logical unit provider”. The logical unit is the device within a target responsible for executing SCSI I/O commands. Perhaps you’ve heard about the acronym LUN in the past. LUN is the Logical Unit Number, a number designating a certain logical unit.
Between the both components is the
stmf,the SCSI target mode framework. It connects the port provider and the logical unit provider.
What’s the advantage: You just have to develop the SCSI target once and for new protocols you just have to develop a relatively simple port provider. This is one of the reasons why we see new port providers quite frequently. In essence this is the reason why the administrative model has changed with the introduction of COMSTAR: The logical unit, the port and the logic to combine both are separate entities thus you configure them separately.
Okay, let’s look how you configure this stuff. I will use two systems in this example:
target has to use a recent of OpenSolaris. You can use OpenSolaris as well as Opensolaris Community Edition. My testbed used OpenSolaris Developer Build 127, but it should work with any reasonable recent version. On the system
initiator any Solaris system will do it, as COMSTAR is a change on the target side, it doesn’t change anything on the initiator part.
Preparing the target system
The COMSTAR framework isn’t installed per default on OpenSolaris, thus you have to install it on your own. But there is an easy way to install all the components to make a storage server out of your installation. Doing so installs much more than you need for iSCSI, but on the other side it installs all other components, when you want to make a CIFS/iSCSI/NFS/MDNP/etc-capable storage server on your system.
Please reboot the system after this step. This framework has many connections to the rest of the system and it’s just easier to reboot after the initial install.
Okay, right after the boot the service
stmf is disabled.
We need this service, so we enable it.
Now we can check the successful startup of the
Obviously we need a backing store for our iSCSI target. In this case i will use a spare provisioned 10 Terabyte emulated ZFS volume.
There are other options besides an emulated ZFS volume like a pregenerated file (
mkfile 10T backingstore), a file that grows from zero to a certain preconfigured file size (
touch backingstore) and finally just mapping an physical device through the system.
Now we have to configure a logical unit provider in the COMSTAR framework to use our backing store. We want a disk, so we use the disk logical unit provider. We have to use the
sbdadm command for this task.
Let’s do a short check, if everything went well.
stmfadm command you can get some additional insight to your newly created logical unit.
But at the moment your brand new LU isn’t visible. You have to add it to an entity called view. In a view you can define which targets a certain initiator can see. For example you can configure with this views, that all your SAP systems just see the SAP logical units and all MS Exchange Servers just see the Exchange logical units on your storage service. Other logical units aren’t visible to the system and they are totally unaware of their existence. For people with a storage background: This is somewhat similar to the LUN masking part ;)
But for this tutorial i will use a simple configuration. I will not impose any access control to this logical unit :
Let’s check the configuration:
However, the logical unit is still not visible to the outside, as we haven’t configured a port provider. The port provider is the component that provides access to the logical unit via iSCSI or FC for example.
Configuring an iSCSI target
So let’s start with the configuration of the iSCSI target. This is the first step where the commands have something to do with iSCSI:
itadm is the tool to configure an iSCSI target. At first we configure a target portal group. A target portal is an IP and a portnumber you use to access an iSCSI target. Target portal groups are used to bundle such portals to ease the configuration of access controls.
Okay, now we configure an iSCSI target entity on this portal group:
This IQN uniquely identifies the target in a network. We will use it from now on when we address this iSCSI target. That’s all. For unauthenticated iSCSI this is all you have to do on the target side: We’ve configured the logical unit provider and enabled a port provider to give other system access to the target with iSCSI.
Configuring the initator without authentication
Okay … now we log into the initiator. Just a quick check …
Just our boot disk. At first we have to install the iSCSI initator. This can be done by
pkg install iscsi.After this step you have to reboot the system.
Okay, the iSCSI service runs on the host
initiator. Now we have to configure the initiator to discover possible logical units on our target:
At first we told the iSCSI initiator to discover logical units at the target portal on
192.168.56.102:3260. After this step, we configured the iSCSI initiator to use the SendTargets method to discover logical units on the other side. The SendTarget command is a command the initiator sends to the configured hosts to gather all targets available to the initiator issuing the SendTarget command. Every iSCSI target implementation has to support the SendTarget command. This is specified by Appendix D of the RFC 3720 This makes the configuration of the iSCSI easier, as you don’t have to configure all the targets manually. Okay, let’s check if the configuration of the discovery was successful:
Now let’s look, what targets were discovered by the iSCSI initiator:
The iSCSI target IQN you saw at the time when you configured the target on the host
target reappears here. Okay, now we have to make all discovered logical units for the use as block devices. In Solaris you use the
devfsadm command for this task. This will populate the
When we start format again, we will see that our host
initiator has a new block device:
A brand new 10 TB iSCSI volume. Of course we can use it for zfs again:
Works as designed. A nice 10 TB ZFS pool on
Configuring the initator with authentication
Okay, in the non-COMSTAR iSCSI tutorial i explained how to use authentication. With authentication the iSCSI target and the iSCSI initiator can check if the host on the other side of the connection is really the expected node. This works with the CHAP mechanism. Perhaps you know it from PPP for your internet access. It works on the foundation of shared secrets. Only when both partners of a communication know the same secret, the communication partner is authentic. It similar to a secret handshake between two peoples.
Okay, let’s configure a bidirectional secret handshake. Just as a good admin we export the pool on it, as the following configuration will terminate the iSCSI connection for a moment.
Okay, to configure the authentication we need to pieces of information at first. Log into the system
target to gather the iqn of the target.
Now log into the system
initiator to gather the IQN of the initiator.
At first we configure the iSCSI target in a way, that it uses chap authentication. We need the IQN of the target here.
Now we have to configure the secrets. The CHAP secrets have be at last 12 characters. At first we set configure the secret, that the initiator will use to authenticate at the target.
Now we configure the secret that the target uses to authenticate itself at the initiator.
Okay, we have to something similar on the system
At first we have to configure the CHAP secret that the initiator uses to authenticate itself
Now we tell the system that this initiator should use CHAP authentication to authenticate a target.
The next steps configure the authentication relation between our initiator and a defined target. At we activate an bi-directional authentication. So the initiator has to authenticate at the target as well as the target has to authenticate itself at the initiator.
Now we tell the iSCSI initator that the iSCSI target uses CHAP to authenticate the ISCSI initiator.
In a last step we configure the shared secret, that the target uses to authenticate the initiator.
Perhaps this figure explain the relation between the secrets better than just command lines can do:
Now we are done. We can do a quick check if our configuration found it’s way to the system. At first a quick look at the iSCSI Target on
Now we do the same check on
Okay, everything is okay … now let’s just reimport the testpool
We are back in business.
The configuration of an iSCSI Target isn’t more difficult than before. It’s just different. I hope i gave you some good insight into this topic.
Do you want to learn more?
docs.sun.com: sbdadm(1M) - SCSI Block Disk command line interface
docs.sun.com: <a href=”http://docs.sun.com/app/docs/doc/819-2240/itadm-1m?a=view”</a>itadm(1M) - administer iSCSI targets
docs.sun.com: iscsiadm(1M) - enable management of iSCSI initiators Tutorials
c0t0d0s0.org : Less known Solaris Features: iSCSI - tutorial for the userland iSCSI implementation in Solaris.