469 lines
20 KiB
Plaintext
469 lines
20 KiB
Plaintext
Introduction
|
|
============
|
|
|
|
The goal of this debug feature is to provide a reliable, responsive,
|
|
accurate and secure debug capability to developers interested in
|
|
debugging MSM subsystem processor images without the use of a hardware
|
|
debugger.
|
|
|
|
The Debug Agent along with the Remote Debug Driver implements a shared
|
|
memory based transport mechanism that allows for a debugger (ex. GDB)
|
|
running on a host PC to communicate with a remote stub running on
|
|
peripheral subsystems such as the ADSP, MODEM etc.
|
|
|
|
The diagram below depicts end to end the components involved to
|
|
support remote debugging:
|
|
|
|
|
|
: :
|
|
: HOST (PC) : MSM
|
|
: ,--------, : ,-------,
|
|
: | | : | Debug | ,--------,
|
|
: |Debugger|<--:-->| Agent | | Remote |
|
|
: | | : | App | +----->| Debug |
|
|
: `--------` : |-------| ,--------, | | Stub |
|
|
: : | Remote| | |<---+ `--------`
|
|
: : | Debug |<-->|--------|
|
|
: : | Driver| | |<---+ ,--------,
|
|
: : `-------` `--------` | | Remote |
|
|
: : LA Shared +----->| Debug |
|
|
: : Memory | Stub |
|
|
: : `--------`
|
|
: : Peripheral Subsystems
|
|
: : (ADSP, MODEM, ...)
|
|
|
|
|
|
Debugger: Debugger application running on the host PC that
|
|
communicates with the remote stub.
|
|
Examples: GDB, LLDB
|
|
|
|
Debug Agent: Software that runs on the Linux Android platform
|
|
that provides connectivity from the MSM to the
|
|
host PC. This involves two portions:
|
|
1) User mode Debug Agent application that discovers
|
|
processes running on the subsystems and creates
|
|
TCP/IP sockets for the host to connect to. In addition
|
|
to this, it creates an info (or meta) port that
|
|
users can connect to discover the various
|
|
processes and their corresponding debug ports.
|
|
|
|
Remote Debug A character based driver that the Debug
|
|
Driver: Agent uses to transport the payload received from the
|
|
host to the debug stub running on the subsystem
|
|
processor over shared memory and vice versa.
|
|
|
|
Shared Memory: Shared memory from the SMEM pool that is accessible
|
|
from the Applications Processor (AP) and the
|
|
subsystem processors.
|
|
|
|
Remote Debug Privileged code that runs in the kernels of the
|
|
Stub: subsystem processors that receives debug commands
|
|
from the debugger running on the host and
|
|
acts on these commands. These commands include reading
|
|
and writing to registers and memory belonging to the
|
|
subsystem's address space, setting breakpoints,
|
|
single stepping etc.
|
|
|
|
Hardware description
|
|
====================
|
|
|
|
The Remote Debug Driver interfaces with the Remote Debug stubs
|
|
running on the subsystem processors and does not drive or
|
|
manage any hardware resources.
|
|
|
|
Software description
|
|
====================
|
|
|
|
The debugger and the remote stubs use Remote Serial Protocol (RSP)
|
|
to communicate with each other. This is widely used protocol by both
|
|
software and hardware debuggers. RSP is an ASCII based protocol
|
|
and used when it is not possible to run GDB server on the target under
|
|
debug.
|
|
|
|
The Debug Agent application along with the Remote Debug Driver
|
|
is responsible for establishing a bi-directional connection from
|
|
the debugger application running on the host to the remote debug
|
|
stub running on a subsystem. The Debug Agent establishes connectivity
|
|
to the host PC via TCP/IP sockets.
|
|
|
|
This feature uses ADB port forwarding to establish connectivity
|
|
between the debugger running on the host and the target under debug.
|
|
|
|
Please note the Debug Agent does not expose HLOS memory to the
|
|
remote subsystem processors.
|
|
|
|
Design
|
|
======
|
|
|
|
Here is the overall flow:
|
|
|
|
1) When the Debug Agent application starts up, it opens up a shared memory
|
|
based transport channel to the various subsystem processor images.
|
|
|
|
2) The Debug Agent application sends messages across to the remote stubs
|
|
to discover the various processes that are running on the subsystem and
|
|
creates debug sockets for each of them.
|
|
|
|
3) Whenever a process running on a subsystem exits, the Debug Agent
|
|
is notified by the stub so that the debug port and other resources
|
|
can be reclaimed.
|
|
|
|
4) The Debug Agent uses the services of the Remote Debug Driver to
|
|
transport payload from the host debugger to the remote stub and vice versa.
|
|
|
|
5) Communication between the Remote Debug Driver and the Remote Debug stub
|
|
running on the subsystem processor is done over shared memory (see figure).
|
|
SMEM services are used to allocate the shared memory that will
|
|
be readable and writeable by the AP and the subsystem image under debug.
|
|
|
|
A separate SMEM allocation takes place for each subsystem processor
|
|
involved in remote debugging. The remote stub running on each of the
|
|
subsystems allocates a SMEM buffer using a unique identifier so that both
|
|
the AP and subsystem get the same physical block of memory. It should be
|
|
noted that subsystem images can be restarted at any time.
|
|
However, when a subsystem comes back up, its stub uses the same unique
|
|
SMEM identifier to allocate the SMEM block. This would not result in a
|
|
new allocation rather the same block of memory in the first bootup instance
|
|
is provided back to the stub running on the subsystem.
|
|
|
|
An 8KB chunk of shared memory is allocated and used for communication
|
|
per subsystem. For multi-process capable subsystems, 16KB chunk of shared
|
|
memory is allocated to allow for simultaneous debugging of more than one
|
|
process running on a single subsystem.
|
|
|
|
The shared memory is used as a circular ring buffer in each direction.
|
|
Thus we have a bi-directional shared memory channel between the AP
|
|
and a subsystem. We call this SMQ. Each memory channel contains a header,
|
|
data and a control mechanism that is used to synchronize read and write
|
|
of data between the AP and the remote subsystem.
|
|
|
|
Overall SMQ memory view:
|
|
:
|
|
: +------------------------------------------------+
|
|
: | SMEM buffer |
|
|
: |-----------------------+------------------------|
|
|
: |Producer: LA | Producer: Remote |
|
|
: |Consumer: Remote | subsystem |
|
|
: | subsystem | Consumer: LA |
|
|
: | | |
|
|
: | Producer| Consumer|
|
|
: +-----------------------+------------------------+
|
|
: | |
|
|
: | |
|
|
: | +--------------------------------------+
|
|
: | |
|
|
: | |
|
|
: v v
|
|
: +--------------------------------------------------------------+
|
|
: | Header | Data | Control |
|
|
: +-----------+---+---+---+-----+----+--+--+-----+---+--+--+-----+
|
|
: | | b | b | b | | S |n |n | | S |n |n | |
|
|
: | Producer | l | l | l | | M |o |o | | M |o |o | |
|
|
: | Ver | o | o | o | | Q |d |d | | Q |d |d | |
|
|
: |-----------| c | c | c | ... | |e |e | ... | |e |e | ... |
|
|
: | | k | k | k | | O | | | | I | | | |
|
|
: | Consumer | | | | | u |0 |1 | | n |0 |1 | |
|
|
: | Ver | 0 | 1 | 2 | | t | | | | | | | |
|
|
: +-----------+---+---+---+-----+----+--+--+-----+---+--+--+-----+
|
|
: | |
|
|
: + |
|
|
: |
|
|
: +------------------------+
|
|
: |
|
|
: v
|
|
: +----+----+----+----+
|
|
: | SMQ Nodes |
|
|
: |----|----|----|----|
|
|
: Node # | 0 | 1 | 2 | ...|
|
|
: |----|----|----|----|
|
|
: Starting Block Index # | 0 | 3 | 8 | ...|
|
|
: |----|----|----|----|
|
|
: # of blocks | 3 | 5 | 1 | ...|
|
|
: +----+----+----+----+
|
|
:
|
|
|
|
Header: Contains version numbers for software compatibility to ensure
|
|
that both producers and consumers on the AP and subsystems know how to
|
|
read from and write to the queue.
|
|
Both the producer and consumer versions are 1.
|
|
: +---------+-------------------+
|
|
: | Size | Field |
|
|
: +---------+-------------------+
|
|
: | 1 byte | Producer Version |
|
|
: +---------+-------------------+
|
|
: | 1 byte | Consumer Version |
|
|
: +---------+-------------------+
|
|
|
|
|
|
Data: The data portion contains multiple blocks [0..N] of a fixed size.
|
|
The block size SM_BLOCKSIZE is fixed to 128 bytes for header version #1.
|
|
Payload sent from the debug agent app is split (if necessary) and placed
|
|
in these blocks. The first data block is placed at the next 8 byte aligned
|
|
address after the header.
|
|
|
|
The number of blocks for a given SMEM allocation is derived as follows:
|
|
Number of Blocks = ((Total Size - Alignment - Size of Header
|
|
- Size of SMQIn - Size of SMQOut)/(SM_BLOCKSIZE))
|
|
|
|
The producer maintains a private block map of each of these blocks to
|
|
determine which of these blocks in the queue is available and which are free.
|
|
|
|
Control:
|
|
The control portion contains a list of nodes [0..N] where N is number
|
|
of available data blocks. Each node identifies the data
|
|
block indexes that contain a particular debug message to be transferred,
|
|
and the number of blocks it took to hold the contents of the message.
|
|
|
|
Each node has the following structure:
|
|
: +---------+-------------------+
|
|
: | Size | Field |
|
|
: +---------+-------------------+
|
|
: | 2 bytes |Staring Block Index|
|
|
: +---------+-------------------+
|
|
: | 2 bytes |Number of Blocks |
|
|
: +---------+-------------------+
|
|
|
|
The producer and the consumer update different parts of the control channel
|
|
(SMQOut / SMQIn) respectively. Each of these control data structures contains
|
|
information about the last node that was written / read, and the actual nodes
|
|
that were written/read.
|
|
|
|
SMQOut Structure (R/W by producer, R by consumer):
|
|
: +---------+-------------------+
|
|
: | Size | Field |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Magic Init Number |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Reset |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Last Sent Index |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Index Free Read |
|
|
: +---------+-------------------+
|
|
|
|
SMQIn Structure (R/W by consumer, R by producer):
|
|
: +---------+-------------------+
|
|
: | Size | Field |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Magic Init Number |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Reset ACK |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Last Read Index |
|
|
: +---------+-------------------+
|
|
: | 4 bytes | Index Free Write |
|
|
: +---------+-------------------+
|
|
|
|
Magic Init Number:
|
|
Both SMQ Out and SMQ In initialize this field with a predefined magic
|
|
number so as to make sure that both the consumer and producer blocks
|
|
have fully initialized and have valid data in the shared memory control area.
|
|
Producer Magic #: 0xFF00FF01
|
|
Consumer Magic #: 0xFF00FF02
|
|
|
|
SMQ Out's Last Sent Index and Index Free Read:
|
|
Only a producer can write to these indexes and they are updated whenever
|
|
there is new payload to be inserted into the SMQ in order to be sent to a
|
|
consumer.
|
|
|
|
The number of blocks required for the SMQ allocation is determined as:
|
|
(payload size + SM_BLOCKSIZE - 1) / SM_BLOCKSIZE
|
|
|
|
The private block map is searched for a large enough continuous set of blocks
|
|
and the user data is copied into the data blocks.
|
|
|
|
The starting index of the free block(s) is updated in the SMQOut's Last Sent
|
|
Index. This update keeps track of which index was last written to and the
|
|
producer uses it to determine where the the next allocation could be done.
|
|
|
|
Every allocation, a producer updates the Index Free Read from its
|
|
collaborating consumer's Index Free Write field (if they are unequal).
|
|
This index value indicates that the consumer has read all blocks associated
|
|
with allocation on the SMQ and that the producer can reuse these blocks for
|
|
subsquent allocations since this is a circular queue.
|
|
|
|
At cold boot and restart, these indexes are initialized to zero and all
|
|
blocks are marked as available for allocation.
|
|
|
|
SMQ In's Last Read Index and Index Free Write:
|
|
These indexes are written to only by a consumer and are updated whenever
|
|
there is new payload to be read from the SMQ. The Last Read Index keeps
|
|
track of which index was last read by the consumer and using this, it
|
|
determines where the next read should be done.
|
|
After completing a read, Last Read Index is incremented to the
|
|
next block index. A consumer updates Index Free Write to the starting
|
|
index of an allocation whenever it has completed processing the blocks.
|
|
This is an optimization that can be used to prevent an additional copy
|
|
of data from the queue into a client's data buffer and the data in the queue
|
|
itself can be used.
|
|
Once Index Free Write is updated, the collaborating producer (on the next
|
|
data allocation) reads the updated Index Free Write value and it then
|
|
updates its corresponding SMQ Out's Index Free Read and marks the blocks
|
|
associated with that index as available for allocation. At cold boot and
|
|
restart, these indexes are initialized to zero.
|
|
|
|
SMQ Out Reset# and SMQ In Reset ACK #:
|
|
Since subsystems can restart at anytime, the data blocks and control channel
|
|
can be in an inconsistent state when a producer or consumer comes up.
|
|
We use Reset and Reset ACK to manage this. At cold boot, the producer
|
|
initializes the Reset# to a known number ex. 1. Every other reset that the
|
|
producer undergoes, the Reset#1 is simply incremented by 1. All the producer
|
|
indexes are reset.
|
|
When the producer notifies the consumer of data availability, the consumer
|
|
reads the producers Reset # and copies that into its SMQ In Reset ACK#
|
|
field when they differ. When that occurs, the consumer resets its
|
|
indexes to 0.
|
|
|
|
6) Asynchronous notifications between a producer and consumer are
|
|
done using the SMP2P service which is interrupt based.
|
|
|
|
Power Management
|
|
================
|
|
|
|
None
|
|
|
|
SMP/multi-core
|
|
==============
|
|
|
|
The driver uses completion to wake up the Debug Agent client threads.
|
|
|
|
Security
|
|
========
|
|
|
|
From the perspective of the subsystem, the AP is untrusted. The remote
|
|
stubs consult the secure debug fuses to determine whether or not the
|
|
remote debugging will be enabled at the subsystem.
|
|
|
|
If the hardware debug fuses indicate that debugging is disabled, the
|
|
remote stubs will not be functional on the subsystem. Writes to the
|
|
queue will only be done if the driver sees that the remote stub has been
|
|
initialized on the subsystem.
|
|
|
|
Therefore even if any untrusted software running on the AP requests
|
|
the services of the Remote Debug Driver and inject RSP messages
|
|
into the shared memory buffer, these RSP messages will be discarded and
|
|
an appropriate error code will be sent up to the invoking application.
|
|
|
|
Performance
|
|
===========
|
|
|
|
During operation, the Remote Debug Driver copies RSP messages
|
|
asynchronously sent from the host debugger to the remote stub and vice
|
|
versa. The debug messages are ASCII based and relatively short
|
|
(<25 bytes) and may once in a while go up to a maximum 700 bytes
|
|
depending on the command the user requested. Thus we do not
|
|
anticipate any major performance impact. Moreover, in a typical
|
|
functional debug scenario performance should not be a concern.
|
|
|
|
Interface
|
|
=========
|
|
|
|
The Remote Debug Driver is a character based device that manages
|
|
a piece of shared memory that is used as a bi-directional
|
|
single producer/consumer circular queue using a next fit allocator.
|
|
Every subsystem, has its own shared memory buffer that is managed
|
|
like a separate device.
|
|
|
|
The driver distinguishes each subsystem processor's buffer by
|
|
registering a node with a different minor number.
|
|
|
|
For each subsystem that is supported, the driver exposes a user space
|
|
interface through the following node:
|
|
- /dev/rdbg-<subsystem>
|
|
Ex. /dev/rdbg-adsp (for the ADSP subsystem)
|
|
|
|
The standard open(), close(), read() and write() API set is
|
|
implemented.
|
|
|
|
The open() syscall will fail if a subsystem is not present or supported
|
|
by the driver or a shared memory buffer cannot be allocated for the
|
|
AP - subsystem communication. It will also fail if the subsytem has
|
|
not initialized the queue on its side. Here are the error codes returned
|
|
in case a call to open() fails:
|
|
ENODEV - memory was not yet allocated for the device
|
|
EEXIST - device is already opened
|
|
ENOMEM - SMEM allocation failed
|
|
ECOMM - Subsytem queue is not yet setup
|
|
ENOMEM - Failure to initialize SMQ
|
|
|
|
read() is a blocking call that will return with the number of bytes written
|
|
by the subsystem whenever the subsystem sends it some payload. Here are the
|
|
error codes returned in case a call to read() fails:
|
|
EINVAL - Invalid input
|
|
ENODEV - Device has not been opened yet
|
|
ERESTARTSYS - call to wait_for_completion_interruptible is interrupted
|
|
ENODATA - call to smq_receive failed
|
|
|
|
write() attempts to send user mode payload out to the subsystem. It can fail
|
|
if the SMQ is full. The number of bytes written is returned back to the user.
|
|
Here are the error codes returned in case a call to write() fails:
|
|
EINVAL - Invalid input
|
|
ECOMM - SMQ send failed
|
|
|
|
In the close() syscall, the control information state of the SMQ is
|
|
initialized to zero thereby preventing any further communication between
|
|
the AP and the subsystem. Here is the error code returned in case
|
|
a call to close() fails:
|
|
ENODEV - device wasn't opened/initialized
|
|
|
|
The Remote Debug driver uses SMP2P for bi-directional AP to subsystem
|
|
notification. Notifications are sent to indicate that there are new
|
|
debug messages available for processing. Each subsystem that is
|
|
supported will need to add a device tree entry per the usage
|
|
specification of SMP2P driver.
|
|
|
|
In case the remote stub becomes non operational or the security configuration
|
|
on the subsystem does not permit debugging, any messages put in the SMQ will
|
|
not be responded to. It is the responsibility of the Debug Agent app and the
|
|
host debugger application such as GDB to timeout and notify the user of the
|
|
non availability of remote debugging.
|
|
|
|
Driver parameters
|
|
=================
|
|
|
|
None
|
|
|
|
Config options
|
|
==============
|
|
|
|
The driver is configured with a device tree entry to map an SMP2P entry
|
|
to the device. The SMP2P entry name used is "rdbg". Please see
|
|
kernel\Documentation\arm\msm\msm_smp2p.txt for information about the
|
|
device tree entry required to configure SMP2P.
|
|
|
|
The driver uses the SMEM allocation type SMEM_LC_DEBUGGER to allocate memory
|
|
for the queue that is used to share data with the subsystems.
|
|
|
|
Dependencies
|
|
============
|
|
|
|
The Debug Agent driver requires services of SMEM to
|
|
allocate shared memory buffers.
|
|
|
|
SMP2P is used as a bi-directional notification
|
|
mechanism between the AP and a subsystem processor.
|
|
|
|
User space utilities
|
|
====================
|
|
|
|
This driver is meant to be used in conjunction with the user mode
|
|
Remote Debug Agent application.
|
|
|
|
Other
|
|
=====
|
|
|
|
None
|
|
|
|
Known issues
|
|
============
|
|
For targets with an external subsystem, we cannot use
|
|
shared memory for communication and would have to use the prevailing
|
|
transport mechanisms that exists between the AP and the external subsystem.
|
|
|
|
This driver cannot be leveraged for such targets.
|
|
|
|
To do
|
|
=====
|
|
|
|
None
|