Intel allocator

Intel allocator — igt implementation of allocator

Functions

Types and Values

Includes

#include <intel_allocator.h>

Description

Introduction

With the era of discrete cards we requested to adopt IGT to handle addresses in userspace only (softpin, without support of relocations). Writing an allocator for single purpose would be relatively easy but supporting different tests with different requirements became quite complicated task where couple of scenarios may be not covered yet.


Assumptions

  • Allocator has to work in multiprocess / multithread environment.

  • Allocator backend (algorithm) should be plugable. Currently we support SIMPLE (borrowed from Mesa allocator), RELOC (pseudo allocator which returns incremented addresses without checking overlapping) and RANDOM (pseudo allocator which randomize addresses without checking overlapping).

  • Has to integrate in intel-bb (our simpler libdrm replacement used in couple of tests).


Implementation

Single process (allows multiple threads)

For single process we don't need to create dedicated entity (kind of arbiter) to solve allocations. Simple locking over allocator data structure is enough. As basic usage example would be:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
struct object {
     uint32_t handle;
     uint64_t offset;
     uint64_t size;
};

struct object obj1, obj2;
uint64_t ahnd, startp, endp, size = 4096, align = 1 << 13;
int fd = -1;

fd = drm_open_driver(DRIVER_INTEL);
ahnd = intel_allocator_open(fd, 0, INTEL_ALLOCATOR_SIMPLE);

obj1.handle = gem_create(4096);
obj2.handle = gem_create(4096);

// Reserve hole for an object in given address.
// In this example the first possible address.
intel_allocator_get_address_range(ahnd, &startp, &endp);
obj1.offset = startp;
igt_assert(intel_allocator_reserve(ahnd, obj1.handle, size, startp));

// Get the most suitable offset for the object. Preferred way.
obj2.offset = intel_allocator_alloc(ahnd, obj2.handle, size, align);

 ...

// Reserved addresses can be only freed by unreserve.
intel_allocator_unreserve(ahnd, obj1.handle, size, obj1.offset);
intel_allocator_free(ahnd, obj2.handle);

gem_close(obj1.handle);
gem_close(obj2.handle);

  • ahnd is allocator handle (vm space handled by it)

  • we call get_address_range() to get start/end range provided by the allocator (we haven't specified its range in open so allocator code will assume some safe address range - we don't want to exercise some potential HW bugs on the last page)

  • alloc() / free() pair just gets address for gem object proposed by the allocator

  • reserve() / unreserve() pair gives us full control of acquire/return range we're interested in

Multiple processes

When process forks and its child uses same fd vm its address space is also the same. Some coordination - in this case interprocess communication - is required to assign proper addresses for gem objects and avoid collision. Additional thread is spawned for such case to cover child processes needs. It uses some form of communication channel to receive, perform action (alloc, free...) and send response to requesting process. Currently SYSVIPC message queue was chosen for this but it can replaced by other mechanism. Allocation techniques are same as for single process, we just need to wrap such code with:

1
2
3
4
5
intel_allocator_multiprocess_start();

... allocation code (open, close, alloc, free, ...)

intel_allocator_multiprocess_stop();

Calling start() spawns additional allocator thread ready for handling incoming allocation requests (open / close are also requests in that case).

Calling stop() request to stop allocator thread unblocking all pending children (if any).

Functions

intel_allocator_init ()

void
intel_allocator_init (void);

Function initializes the allocators infrastructure. The second call will override current infra and destroy existing there allocators. It is called in igt_constructor.


intel_allocator_multiprocess_start ()

void
intel_allocator_multiprocess_start (void);

Function turns on intel_allocator multiprocess mode what means all allocations from children processes are performed in a separate thread within main igt process. Children are aware of the situation and use some interprocess communication channel to send/receive messages (open, close, alloc, free, ...) to/from allocator thread.

Must be used when you want to use an allocator in non single-process code. All allocations in threads spawned in main igt process are handled by mutexing, not by sending/receiving messages to/from allocator thread.

Note. This destroys all previously created allocators and theirs content.


intel_allocator_multiprocess_stop ()

void
intel_allocator_multiprocess_stop (void);

Function turns off intel_allocator multiprocess mode what means stopping allocator thread and deinitializing its data.


intel_allocator_open ()

uint64_t
intel_allocator_open (int fd,
                      uint32_t ctx,
                      uint8_t allocator_type);

Function opens an allocator instance for given fd and ctx and returns its handle. If the allocator for such pair doesn't exist it is created with refcount = 1. Parallel opens returns same handle bumping its refcount.

Parameters

fd

i915 descriptor

 

ctx

context

 

allocator_type

one of INTEL_ALLOCATOR_* define

 

Returns

unique handle to the currently opened allocator.

Notes: we pass ALLOC_STRATEGY_HIGH_TO_LOW as default, playing with higher addresses makes easier to find addressing issues (like passing non-canonical offsets, which won't be catched unless 47-bit is set).


intel_allocator_open_full ()

uint64_t
intel_allocator_open_full (int fd,
                           uint32_t ctx,
                           uint64_t start,
                           uint64_t end,
                           uint8_t allocator_type,
                           enum allocator_strategy strategy);

Function opens an allocator instance within <start , end ) vm for given fd and ctx and returns its handle. If the allocator for such pair doesn't exist it is created with refcount = 1. Parallel opens returns same handle bumping its refcount.

Parameters

fd

i915 descriptor

 

ctx

context

 

start

address of the beginning

 

end

address of the end

 

allocator_type

one of INTEL_ALLOCATOR_* define

 

strategy

passed to the allocator to define the strategy (like order of allocation, see notes below).

 

Returns

unique handle to the currently opened allocator.

Notes:

If start = end = 0, the allocator is opened for the whole available gtt.

Strategy is generally used internally by the underlying allocator:

For SIMPLE allocator:

  • ALLOC_STRATEGY_HIGH_TO_LOW means topmost addresses are allocated first,

  • ALLOC_STRATEGY_LOW_TO_HIGH opposite, allocation starts from lowest addresses.

For RANDOM allocator:

  • no strategy is currently implemented.


intel_allocator_open_vm ()

uint64_t
intel_allocator_open_vm (int fd,
                         uint32_t vm,
                         uint8_t allocator_type);

intel_allocator_open_vm_full ()

uint64_t
intel_allocator_open_vm_full (int fd,
                              uint32_t vm,
                              uint64_t start,
                              uint64_t end,
                              uint8_t allocator_type,
                              enum allocator_strategy strategy);

intel_allocator_open_vm_as ()

uint64_t
intel_allocator_open_vm_as (uint64_t allocator_handle,
                            uint32_t new_vm);

intel_allocator_close ()

bool
intel_allocator_close (uint64_t allocator_handle);

Function decreases an allocator refcount for the given handle . When refcount reaches zero allocator is closed (destroyed) and all allocated / reserved areas are freed.

Parameters

allocator_handle

handle to the allocator that will be closed

 

Returns

true if closed allocator was empty, false otherwise.


intel_allocator_get_address_range ()

void
intel_allocator_get_address_range (uint64_t allocator_handle,
                                   uint64_t *startp,
                                   uint64_t *endp);

Function fills startp , endp with respectively, starting and ending offset of the allocator working virtual address space range.

Note. Allocators working ranges can differ depending on the device or the allocator type so before reserving a specific offset a good practise is to ensure that address is between accepted range.

Parameters

allocator_handle

handle to an allocator

 

startp

pointer to the variable where function writes starting offset

 

endp

pointer to the variable where function writes ending offset

 

intel_allocator_alloc ()

uint64_t
intel_allocator_alloc (uint64_t allocator_handle,
                       uint32_t handle,
                       uint64_t size,
                       uint64_t alignment);

Same as __intel_allocator_alloc() but asserts if allocator can't return valid address. Uses default allocation strategy chosen during opening the allocator.

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object

 

size

size of an object

 

alignment

determines object alignment

 

intel_allocator_alloc_with_strategy ()

uint64_t
intel_allocator_alloc_with_strategy (uint64_t allocator_handle,
                                     uint32_t handle,
                                     uint64_t size,
                                     uint64_t alignment,
                                     enum allocator_strategy strategy);

Same as __intel_allocator_alloc() but asserts if allocator can't return valid address. Use strategy instead of default chosen during opening the allocator.

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object

 

size

size of an object

 

alignment

determines object alignment

 

strategy

strategy of allocation

 

intel_allocator_free ()

bool
intel_allocator_free (uint64_t allocator_handle,
                      uint32_t handle);

Function free object identified by the handle in allocator what makes it offset again allocable.

Note. Reserved objects can only be freed by an intel_allocator_unreserve function.

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object to be freed

 

Returns

true if the object was successfully freed, otherwise false.


intel_allocator_is_allocated ()

bool
intel_allocator_is_allocated (uint64_t allocator_handle,
                              uint32_t handle,
                              uint64_t size,
                              uint64_t offset);

Function checks whether the object identified by the handle and size is allocated at the offset .

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object

 

size

size of an object

 

offset

address of an object

 

Returns

true if the object is currently allocated at the offset , otherwise false.


intel_allocator_reserve ()

bool
intel_allocator_reserve (uint64_t allocator_handle,
                         uint32_t handle,
                         uint64_t size,
                         uint64_t offset);

Function reserves space that starts at the offset and has size . Optionally we can pass handle to mark that space is for a specific object, otherwise pass -1.

Note. Reserved space is identified by offset and size, not a handle. So an object can have multiple reserved spaces with its handle.

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object

 

size

size of an object

 

offset

address of an object

 

Returns

true if space is successfully reserved, otherwise false.


intel_allocator_unreserve ()

bool
intel_allocator_unreserve (uint64_t allocator_handle,
                           uint32_t handle,
                           uint64_t size,
                           uint64_t offset);

Function unreserves space that starts at the offset , size and handle .

Note. handle , size and offset have to match those used in reservation. i.e. check with the same offset but even smaller size will fail.

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object

 

size

size of an object

 

offset

address of an object

 

Returns

true if the space is successfully unreserved, otherwise false.


intel_allocator_is_reserved ()

bool
intel_allocator_is_reserved (uint64_t allocator_handle,
                             uint64_t size,
                             uint64_t offset);

Function checks whether space starting at the offset and size is currently under reservation.

Note. size and offset have to match those used in reservation, i.e. check with the same offset but even smaller size will fail.

Parameters

allocator_handle

handle to an allocator

 

size

size of an object

 

offset

address of an object

 

Returns

true if space is reserved, othwerise false.


intel_allocator_reserve_if_not_allocated ()

bool
intel_allocator_reserve_if_not_allocated
                               (uint64_t allocator_handle,
                                uint32_t handle,
                                uint64_t size,
                                uint64_t offset,
                                bool *is_allocatedp);

Function checks whether the object identified by the handle and size is allocated at the offset and writes the result to is_allocatedp . If it's not it reserves it at the given offset .

Parameters

allocator_handle

handle to an allocator

 

handle

handle to an object

 

size

size of an object

 

offset

address of an object

 

is_allocatedp

if not NULL function writes there object allocation status (true/false)

 

Returns

true if the space for an object was reserved, otherwise false.


intel_allocator_print ()

void
intel_allocator_print (uint64_t allocator_handle);

Function prints statistics and content of the allocator. Mainly for debugging purposes.

Note. Printing possible only in the main process.

Parameters

allocator_handle

handle to an allocator

 

sign_extend64 ()

return
sign_extend64 (offset Param1);

DECANONICAL()

#define DECANONICAL(offset) (offset & ((1ull << GEN8_GTT_ADDRESS_WIDTH) - 1))

Types and Values

enum allocator_strategy

Members

ALLOC_STRATEGY_NONE

   

ALLOC_STRATEGY_LOW_TO_HIGH

   

ALLOC_STRATEGY_HIGH_TO_LOW

   

struct intel_allocator

struct intel_allocator {
	int fd;
	uint8_t type;
	enum allocator_strategy strategy;
	_Atomic(int32_t) refcount;
	pthread_mutex_t mutex;

	/* allocator's private structure */
	void *priv;

	void (*get_address_range)(struct intel_allocator *ial,
				  uint64_t *startp, uint64_t *endp);
	uint64_t (*alloc)(struct intel_allocator *ial, uint32_t handle,
			  uint64_t size, uint64_t alignment,
			  enum allocator_strategy strategy);
	bool (*is_allocated)(struct intel_allocator *ial, uint32_t handle,
			     uint64_t size, uint64_t offset);
	bool (*reserve)(struct intel_allocator *ial,
			uint32_t handle, uint64_t start, uint64_t end);
	bool (*unreserve)(struct intel_allocator *ial,
			  uint32_t handle, uint64_t start, uint64_t end);
	bool (*is_reserved)(struct intel_allocator *ial,
			    uint64_t start, uint64_t end);
	bool (*free)(struct intel_allocator *ial, uint32_t handle);

	void (*destroy)(struct intel_allocator *ial);

	bool (*is_empty)(struct intel_allocator *ial);

	void (*print)(struct intel_allocator *ial, bool full);
};

ALLOC_INVALID_ADDRESS

#define ALLOC_INVALID_ADDRESS (-1ull)

INTEL_ALLOCATOR_NONE

#define INTEL_ALLOCATOR_NONE   0

INTEL_ALLOCATOR_RELOC

#define INTEL_ALLOCATOR_RELOC  1

INTEL_ALLOCATOR_RANDOM

#define INTEL_ALLOCATOR_RANDOM 2

INTEL_ALLOCATOR_SIMPLE

#define INTEL_ALLOCATOR_SIMPLE 3

GEN8_GTT_ADDRESS_WIDTH

#define GEN8_GTT_ADDRESS_WIDTH 48

shift

	int shift = 63 - high;

do_relocs

	bool do_relocs = gem_has_relocations(fd);