core/vsl/vsl_binary_explicit_io.h File Reference

Byte-swapping, arbitrary length integer conversion, and explicit I/O. More...

#include <vxl_config.h>
#include <vcl_cassert.h>
#include <vcl_cstring.h>
#include <vcl_cstddef.h>
#include <vcl_iostream.h>
#include "vsl_binary_io.h"

Go to the source code of this file.

Defines
#define	VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(size_of_type)   (((size_of_type * 8)/7) + ((((size_of_type * 8) % 7) == 0) ? 0: 1))
	The maximum length of buffer to use with arbitrary length integers.
Functions
void	vsl_swap_bytes (char *ptr, unsigned nbyte, vcl_size_t nelem=1)
	Perform byte swapping in situ.
void	vsl_swap_bytes_to_buffer (const char *source, char *dest, unsigned nbyte, vcl_size_t nelem=1)
	Perform byte swapping to a buffer.
template<class T >
vcl_size_t	vsl_convert_to_arbitrary_length_unsigned_impl (const T *ints, unsigned char *buffer, vcl_size_t count)
	Implement arbitrary length conversion for unsigned integers.
template<class T >
vcl_size_t	vsl_convert_to_arbitrary_length_signed_impl (const T *ints, unsigned char *buffer, vcl_size_t count)
	Implement arbitrary length conversion for signed integers.
template<class T >
vcl_size_t	vsl_convert_from_arbitrary_length_signed_impl (const unsigned char *buffer, T *ints, vcl_size_t count)
	Implement arbitrary length conversion for signed integers.
template<class T >
vcl_size_t	vsl_convert_from_arbitrary_length_unsigned_impl (const unsigned char *buffer, T *ints, vcl_size_t count=1)
	Implement arbitrary length conversion for unsigned integers.
vcl_size_t	vsl_convert_to_arbitrary_length (const unsigned long *ints, unsigned char *buffer, vcl_size_t count=1)
	Encode an array of ints into an arbitrary length format.
vcl_size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, unsigned long *ints, vcl_size_t count=1)
	Decode a buffer of arbitrary length integers.
vcl_size_t	vsl_convert_to_arbitrary_length (const long *ints, unsigned char *buffer, vcl_size_t count=1)
	Encode an array of ints into an arbitrary length format.
vcl_size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, long *ints, vcl_size_t count=1)
	Decode a buffer of arbitrary length integers.
vcl_size_t	vsl_convert_to_arbitrary_length (const unsigned int *ints, unsigned char *buffer, vcl_size_t count=1)
	Encode an array of ints into an arbitrary length format.
vcl_size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, unsigned int *ints, vcl_size_t count=1)
	Decode a buffer of arbitrary length integers.
vcl_size_t	vsl_convert_to_arbitrary_length (const int *ints, unsigned char *buffer, vcl_size_t count=1)
	Encode an array of ints into an arbitrary length format.
vcl_size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, int *ints, vcl_size_t count=1)
	Decode a buffer of arbitrary length integers.
vcl_size_t	vsl_convert_to_arbitrary_length (const unsigned short *ints, unsigned char *buffer, vcl_size_t count=1)
	Encode an array of ints into an arbitrary length format.
vcl_size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, unsigned short *ints, vcl_size_t count=1)
	Decode a buffer of arbitrary length integers.
vcl_size_t	vsl_convert_to_arbitrary_length (const short *ints, unsigned char *buffer, vcl_size_t count=1)
	Encode an array of ints into an arbitrary length format.
vcl_size_t	vsl_convert_from_arbitrary_length (const unsigned char *buffer, short *ints, vcl_size_t count=1)
	Decode a buffer of arbitrary length integers.
void	vsl_b_write_uint_16 (vsl_b_ostream &os, unsigned long n)
	Write an unsigned int as 16 bits to vsl_b_ostream.
void	vsl_b_read_uint_16 (vsl_b_istream &is, unsigned long &n)
	Read an unsigned int as 16 bits from vsl_b_istream.
void	vsl_b_write_int_16 (vsl_b_ostream &os, long n)
	Write a signed int as 16 bits to vsl_b_ostream.
void	vsl_b_read_int_16 (vsl_b_istream &is, long &n)
	Read a signed int as 16 bits from vsl_b_istream.
void	vsl_b_write_uint_64 (vsl_b_ostream &os, vcl_size_t n)
	Write a vcl_size_t as 64 bits to vsl_b_ostream.
void	vsl_b_read_uint_64 (vsl_b_istream &is, vcl_size_t &n)
	Read a vcl_size_t as 64 bits from vsl_b_istream.
Variables
There is a problem with the ENDIAN indication	macros

---

Detailed Description

Byte-swapping, arbitrary length integer conversion, and explicit I/O.

Author:

Ian Scott (Manchester) April 2001

Include this file if you want to perform integer IO using fixed size encoding.

If you want to read or write a large number of floating points, then;

• Floats and doubles need byte swapped, and this can be done in situ.
• Shorts, ints and longs need converted to/from the arbitrary length format.

Implementation details:

The arbitrary length encoding takes the number and breaks it into 7 bit nibbles. Each nibble is saved with the 8th bit set if this is the last byte. The nibbles are stored in little endian order. ie The first byte of the encoded format represents the least significant 7 bits.

The algorithm used to encode an unsigned value works as follows

    while value is greater than 2^7-1
      emit 0 bit
      emit least significant 7 bits of value.
      shift value right 7 bits
    emit bit 1
    emit value embedded in 7 bits
   

Definition in file vsl_binary_explicit_io.h.

---

Define Documentation

#define VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH

(

size_of_type

)

(((size_of_type * 8)/7) + ((((size_of_type * 8) % 7) == 0) ? 0: 1))

The maximum length of buffer to use with arbitrary length integers.

This macro tells you the size of buffer you need when using vsl_convert_ints_to_arbitrary_length(). You should give the macro the size of the type you want to convert. If you are converting several integers at once, multiply the value by the number of integers.

Definition at line 160 of file vsl_binary_explicit_io.h.

---

Function Documentation

void vsl_b_read_int_16	(	vsl_b_istream &	is,
		long &	n
	)		[inline]

Read a signed int as 16 bits from vsl_b_istream.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

This function will only read values saved using vsl_b_write_int_16().

Definition at line 665 of file vsl_binary_explicit_io.h.

void vsl_b_read_uint_16	(	vsl_b_istream &	is,
		unsigned long &	n
	)		[inline]

Read an unsigned int as 16 bits from vsl_b_istream.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

This function will only read values saved using vsl_b_write_uint_16().

Definition at line 632 of file vsl_binary_explicit_io.h.

void vsl_b_read_uint_64	(	vsl_b_istream &	is,
		vcl_size_t &	n
	)		[inline]

Read a vcl_size_t as 64 bits from vsl_b_istream.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

This function will only read values saved using vsl_b_write_uint_64().

Definition at line 705 of file vsl_binary_explicit_io.h.

void vsl_b_write_int_16	(	vsl_b_ostream &	os,
		long	n
	)		[inline]

Write a signed int as 16 bits to vsl_b_ostream.

If your signed int cannot be represented in 16 bits (e.g. on a 32 bit platform) the stream's error flag will be set.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

You must vsl_b_read_int_16() to read the value saved with this function.

Definition at line 651 of file vsl_binary_explicit_io.h.

void vsl_b_write_uint_16	(	vsl_b_ostream &	os,
		unsigned long	n
	)		[inline]

Write an unsigned int as 16 bits to vsl_b_ostream.

If your signed int cannot be represented in 16 bits (e.g. on a 32 bit platform) the stream's error flag will be set.

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

You must use vsl_b_read_uint_16() to read the value saved with this function.

Definition at line 618 of file vsl_binary_explicit_io.h.

void vsl_b_write_uint_64	(	vsl_b_ostream &	os,
		vcl_size_t	n
	)		[inline]

Write a vcl_size_t as 64 bits to vsl_b_ostream.

Will assert if your vcl_size_t cannot be represented in 64 bits (e.g. on some 128 bit platforms).

Warning: This function should be used infrequently and carefully. Under all normal circumstances, the generic vsl_b_read and vsl_b_write in vsl_binary_io.h will be perfectly adequate.

You must use vsl_b_read_uint_64() to read the value saved with this function.

Definition at line 691 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_from_arbitrary_length	(	const unsigned char *	buffer,
		unsigned long *	ints,
		vcl_size_t	count = 1
	)		[inline]

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal longs.

Parameters:

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns:

the number of bytes used, or zero on error.

Definition at line 323 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_from_arbitrary_length	(	const unsigned char *	buffer,
		long *	ints,
		vcl_size_t	count = 1
	)		[inline]

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal longs.

Parameters:

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns:

the number of bytes used, or zero on error.

Definition at line 353 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_from_arbitrary_length	(	const unsigned char *	buffer,
		unsigned int *	ints,
		vcl_size_t	count = 1
	)		[inline]

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters:

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns:

the number of bytes used, or zero on error.

Definition at line 384 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_from_arbitrary_length	(	const unsigned char *	buffer,
		int *	ints,
		vcl_size_t	count = 1
	)		[inline]

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters:

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns:

the number of bytes used, or zero on error.

Definition at line 415 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_from_arbitrary_length	(	const unsigned char *	buffer,
		unsigned short *	ints,
		vcl_size_t	count = 1
	)		[inline]

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters:

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns:

the number of bytes used, or zero on error.

Definition at line 446 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_from_arbitrary_length	(	const unsigned char *	buffer,
		short *	ints,
		vcl_size_t	count = 1
	)		[inline]

Decode a buffer of arbitrary length integers.

Converts from the integers from the arbitrary length format into an array of normal ints.

Parameters:

buffer	The buffer to be converted.
count	Number of integers expected. Cannot be zero.
ints	should point to a buffer at least as long as count.

Returns:

the number of bytes used, or zero on error.

Definition at line 477 of file vsl_binary_explicit_io.h.

template<class T >

vcl_size_t vsl_convert_from_arbitrary_length_signed_impl	(	const unsigned char *	buffer,
		T *	ints,
		vcl_size_t	count
	)		[inline]

Implement arbitrary length conversion for signed integers.

This function should only be used by this header file.

Definition at line 213 of file vsl_binary_explicit_io.h.

template<class T >

vcl_size_t vsl_convert_from_arbitrary_length_unsigned_impl	(	const unsigned char *	buffer,
		T *	ints,
		vcl_size_t	count = 1
	)		[inline]

Implement arbitrary length conversion for unsigned integers.

This function should only be used by this header file.

Definition at line 262 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_to_arbitrary_length	(	const unsigned long *	ints,
		unsigned char *	buffer,
		vcl_size_t	count = 1
	)		[inline]

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(unsigned long)) * count

Definition at line 307 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_to_arbitrary_length	(	const long *	ints,
		unsigned char *	buffer,
		vcl_size_t	count = 1
	)		[inline]

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(long)) * count

Definition at line 337 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_to_arbitrary_length	(	const unsigned int *	ints,
		unsigned char *	buffer,
		vcl_size_t	count = 1
	)		[inline]

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(unsigned int)) * count

Definition at line 367 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_to_arbitrary_length	(	const int *	ints,
		unsigned char *	buffer,
		vcl_size_t	count = 1
	)		[inline]

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(int)) * count

Definition at line 398 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_to_arbitrary_length	(	const unsigned short *	ints,
		unsigned char *	buffer,
		vcl_size_t	count = 1
	)		[inline]

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(unsigned short)) * count

Definition at line 429 of file vsl_binary_explicit_io.h.

vcl_size_t vsl_convert_to_arbitrary_length	(	const short *	ints,
		unsigned char *	buffer,
		vcl_size_t	count = 1
	)		[inline]

Encode an array of ints into an arbitrary length format.

The return value is the number of bytes used. buffer should be at least as long as VSL_MAX_ARBITRARY_INT_BUFFER_LENGTH(sizeof(short)) * count

Definition at line 460 of file vsl_binary_explicit_io.h.

template<class T >

vcl_size_t vsl_convert_to_arbitrary_length_signed_impl	(	const T *	ints,
		unsigned char *	buffer,
		vcl_size_t	count
	)		[inline]

Implement arbitrary length conversion for signed integers.

This function should only be used by this header file. Returns the number of bytes written

Definition at line 191 of file vsl_binary_explicit_io.h.

template<class T >

vcl_size_t vsl_convert_to_arbitrary_length_unsigned_impl	(	const T *	ints,
		unsigned char *	buffer,
		vcl_size_t	count
	)		[inline]

Implement arbitrary length conversion for unsigned integers.

This function should only be used by this header file. Returns the number of bytes written

Definition at line 168 of file vsl_binary_explicit_io.h.

void vsl_swap_bytes	(	char *	ptr,
		unsigned	nbyte,
		vcl_size_t	nelem = 1
	)		[inline]

Perform byte swapping in situ.

Where appropriate, swaps pairs of bytes (behaviour is system dependent) Apply this function to your floating-point data to convert from system format to I/O format. Apply the same function to do the reverse conversion.

Parameters:

ptr	The buffer to be byte-swapped.
nbyte	The length of the fundamental type, e.g. sizeof(float).
nelem	The number of elements in the buffer (default: 1).

The standard I/O format is little-endian. The code assumes that your system's floats and doubles are stored in memory in either little-endian or big-endian IEEE floating point formats.

Note: There is no point in #ifdef-ing out calls to byte-swapping if you are on a little-endian machine. An optimising compiler will inline the function to nothing for little-endian machines anyway.

If your computer doesn't use IEEE format reals, then we really should redesign the floating point IO. Proposed design notes: Should do conversion to and from a buffer, rather than in place, (since size not known in general) double and reals should be converted to IEEE format. Someone needs to write a long double format anyway. Don't forget to fix all the code that calls vsl_swap_bytes. Really should check anything that #includes this file.

Definition at line 80 of file vsl_binary_explicit_io.h.

void vsl_swap_bytes_to_buffer	(	const char *	source,
		char *	dest,
		unsigned	nbyte,
		vcl_size_t	nelem = 1
	)		[inline]

Perform byte swapping to a buffer.

Same as vsl_swap_bytes, but saves the results in a buffer. In general use vsl_swap_bytes where possible, because it is faster.

Definition at line 105 of file vsl_binary_explicit_io.h.

---

Variable Documentation

There is a problem with the ENDIAN indication macros

Definition at line 44 of file vsl_binary_explicit_io.h.