Provided by: libfabric-dev_1.17.0-3ubuntu1_amd64 
NAME
fi_cm - Connection management operations
fi_connect / fi_listen / fi_accept / fi_reject / fi_shutdown
Manage endpoint connection state.
fi_setname / fi_getname / fi_getpeer
Set local, or return local or peer endpoint address.
fi_join / fi_close / fi_mc_addr
Join, leave, or retrieve a multicast address.
SYNOPSIS
#include <rdma/fi_cm.h>
int fi_connect(struct fid_ep *ep, const void *addr,
const void *param, size_t paramlen);
int fi_listen(struct fid_pep *pep);
int fi_accept(struct fid_ep *ep, const void *param, size_t paramlen);
int fi_reject(struct fid_pep *pep, fid_t handle,
const void *param, size_t paramlen);
int fi_shutdown(struct fid_ep *ep, uint64_t flags);
int fi_setname(fid_t fid, void *addr, size_t addrlen);
int fi_getname(fid_t fid, void *addr, size_t *addrlen);
int fi_getpeer(struct fid_ep *ep, void *addr, size_t *addrlen);
int fi_join(struct fid_ep *ep, const void *addr, uint64_t flags,
struct fid_mc **mc, void *context);
int fi_close(struct fid *mc);
fi_addr_t fi_mc_addr(struct fid_mc *mc);
ARGUMENTS
ep / pep
Fabric endpoint on which to change connection state.
fid Active or passive endpoint to get/set address.
addr Buffer to address. On a set call, the endpoint will be assigned the specified address. On a get,
the local address will be copied into the buffer, up to the space provided. For connect, this pa‐
rameter indicates the peer address to connect to. The address must be in the same format as that
specified using fi_info: addr_format when the endpoint was created.
addrlen
On input, specifies size of addr buffer. On output, stores number of bytes written to addr
buffer.
param User-specified data exchanged as part of the connection exchange.
paramlen
Size of param buffer.
info Fabric information associated with a connection request.
mc Multicast group associated with an endpoint.
flags Additional flags for controlling connection operation.
context
User context associated with the request.
DESCRIPTION
Connection management functions are used to connect an connection-oriented endpoint to a peer endpoint.
fi_listen
The fi_listen call indicates that the specified endpoint should be transitioned into a passive connection
state, allowing it to accept incoming connection requests. Connection requests against a listening end‐
point are reported asynchronously to the user through a bound CM event queue using the FI_CONNREQ event
type. The number of outstanding connection requests that can be queued at an endpoint is limited by the
listening endpoint’s backlog parameter. The backlog is initialized based on administrative configuration
values, but may be adjusted through the fi_control call.
fi_connect
The fi_connect call initiates a connection request on a connection-oriented endpoint to the destination
address. fi_connect may only be called on an endpoint once in its lifetime.
fi_accept / fi_reject
The fi_accept and fi_reject calls are used on the passive (listening) side of a connection to accept or
reject a connection request, respectively. To accept a connection, the listening application first waits
for a connection request event (FI_CONNREQ). After receiving such an event, the application allocates a
new endpoint to accept the connection. This endpoint must be allocated using an fi_info structure refer‐
encing the handle from this FI_CONNREQ event. fi_accept is then invoked with the newly allocated end‐
point. If the listening application wishes to reject a connection request, it calls fi_reject with the
listening endpoint and a reference to the connection request.
A successfully accepted connection request will result in the active (connecting) endpoint seeing an
FI_CONNECTED event on its associated event queue. A rejected or failed connection request will generate
an error event. The error entry will provide additional details describing the reason for the failed at‐
tempt.
An FI_CONNECTED event will also be generated on the passive side for the accepting endpoint once the con‐
nection has been properly established. The fid of the FI_CONNECTED event will be that of the endpoint
passed to fi_accept as opposed to the listening passive endpoint. Outbound data transfers cannot be ini‐
tiated on a connection-oriented endpoint until an FI_CONNECTED event has been generated. However, re‐
ceive buffers may be associated with an endpoint anytime.
fi_shutdown
The fi_shutdown call is used to gracefully disconnect an endpoint from its peer. The flags parameter is
reserved and must be 0.
Outstanding operations posted to the endpoint when fi_shutdown is called will be canceled or discarded.
Notification of canceled operations will be reported by the provider to the corresponding completion
queue(s). Discarded operations will silently be dropped, with no completions generated. The choice of
canceling, versus discarding operations, is provider dependent. However, all canceled completions will
be written before fi_shutdown returns.
When called, fi_shutdown does not affect completions already written to a completion queue. Any queued
completions associated with asynchronous operations posted to the endpoint may still be retrieved from
the corresponding completion queue(s) after an endpoint has been shutdown.
An FI_SHUTDOWN event will be generated for an endpoint when the remote peer issues a disconnect using
fi_shutdown or abruptly closes the endpoint. Note that in the abrupt close case, an FI_SHUTDOWN event
will only be generated if the peer system is reachable and a service or kernel agent on the peer system
is able to notify the local endpoint that the connection has been aborted.
fi_close
Fi_close is used to disassociate an endpoint from a multicast group and close all resources associated
with the group. Fi_close must be called on all multicast groups that an endpoint joins.
fi_setname
The fi_setname call may be used to modify or assign the address of the local endpoint. It is conceptual‐
ly similar to the socket bind operation. An endpoint may be assigned an address on its creation, through
the fi_info structure. The fi_setname call allows an endpoint to be created without being associated
with a specific service (e.g., port number) and/or node (e.g., network) address, with the addressing as‐
signed dynamically. The format of the specified addressing data must match that specified through the
fi_info structure when the endpoint was created.
If no service address is specified and a service address has not yet been assigned to the endpoint, then
the provider will allocate a service address and assign it to the endpoint. If a node or service address
is specified, then, upon successful completion of fi_setname, the endpoint will be assigned the given ad‐
dressing. If an address cannot be assigned, or the endpoint address cannot be modified, an appropriate
fabric error number is returned.
fi_getname / fi_getpeer
The fi_getname and fi_getpeer calls may be used to retrieve the local or peer endpoint address, respec‐
tively. On input, the addrlen parameter should indicate the size of the addr buffer. If the actual ad‐
dress is larger than what can fit into the buffer, it will be truncated and -FI_ETOOSMALL will be re‐
turned. On output, addrlen is set to the size of the buffer needed to store the address, which may be
larger than the input value.
fi_getname is not guaranteed to return a valid source address until after the specified endpoint has been
enabled or has had an address assigned. An endpoint may be enabled explicitly through fi_enable, or im‐
plicitly, such as through fi_connect or fi_listen. An address may be assigned using fi_setname. fi_get‐
peer is not guaranteed to return a valid peer address until an endpoint has been completely connected –
an FI_CONNECTED event has been generated.
fi_join
This call attaches an endpoint to a multicast group. By default, the endpoint will join the group based
on the data transfer capabilities of the endpoint. For example, if the endpoint has been configured to
both send and receive data, then the endpoint will be able to initiate and receive transfers to and from
the multicast group. The fi_join flags may be used to restrict access to the multicast group, subject to
endpoint capability limitations.
Multicast join operations complete asynchronously. An endpoint must be bound to an event queue prior to
calling fi_join. The result of the join operation will be reported to the EQ as an FI_JOIN_COMPLETE
event. Applications cannot issue multicast transfers until receiving notification that the join opera‐
tion has completed. Note that an endpoint may begin receiving messages from the multicast group as soon
as the join completes, which can occur prior to the FI_JOIN_COMPLETE event being generated.
Applications must call fi_close on the multicast group to disconnect the endpoint from the group. After
a join operation has completed, the fi_mc_addr call may be used to retrieve the address associated with
the multicast group.
fi_mc_addr
Returns the fi_addr_t address associated with a multicast group. This address must be used when trans‐
mitting data to a multicast group and paired with the FI_MULTICAST operation flag.
FLAGS
Except in functions noted below, flags are reserved and must be 0.
FI_SEND
Applies to fi_join. This flag indicates that the endpoint should join the multicast group as a
send only member. The endpoint must be configured for transmit operations to use this flag, or an
error will occur.
FI_RECV
Applies to fi_join. This flag indicates that the endpoint should join the multicast group with
receive permissions only. The endpoint must be configured for receive operations to use this
flag, or an error will occur.
RETURN VALUE
Returns 0 on success. On error, a negative value corresponding to fabric errno is returned. Fabric er‐
rno values are defined in rdma/fi_errno.h.
ERRORS
NOTES
For connection-oriented endpoints, the buffer referenced by param will be sent as part of the connection
request or response, subject to the constraints of the underlying connection protocol. Applications may
use fi_getopt with the FI_OPT_CM_DATA_SIZE endpoint option to determine the size of application data that
may be exchanged as part of a connection request or response. The fi_connect, fi_accept, and fi_reject
calls will silently truncate any application data which cannot fit into underlying protocol messages.
User data exchanged as part of the connection process is available as part of the fi_eq_cm_entry struc‐
ture, for FI_CONNREQ and FI_CONNECTED events, or as additional err_data to fi_eq_err_entry, in the case
of a rejected connection.
SEE ALSO
fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_eq(3)
AUTHORS
OpenFabrics.
Libfabric Programmer’s Manual 2022-12-11 fi_cm(3)