/ 
ExtUtils-DynaLib-0.30
/ ExtUtils::DynaLib

NAME

ExtUtils::DynaLib - Perl extension for calling dynamically loaded C functions

SYNOPSIS

  use ExtUtils::DynaLib;
  $lib = new ExtUtils::DynaLib( $linker_arg );

  $func = $lib->DeclareSub( $symbol_name
			[, $return_type [, @arg_types] ] );
  # or
  $func = $lib->DeclareSub( { "name"    => $symbol_name,
			["return" => $return_type,]
			["args"   => \@arg_types,]
			["decl"   => $decl,]
			} );
  $result = $func->( @args );

  use ExtUtils::DynaLib qw(DeclareSub);
  $func = DeclareSub( $function_pointer,
			[, $return_type [, @arg_types] ] );
  # or
  $func = DeclareSub( { "ptr" => $function_pointer,
			["return" => $return_type,]
			["args"   => \@arg_types,]
			["decl"   => $decl,]
			["libref" => $libref,]
			} );
  $result = $func->( @args );

  $callback = new ExtUtils::DynaLib::Callback( \&my_sub,
			$return_type, @arg_types );
  $callback_pointer = $callback->Ptr();

DESCRIPTION

This module allows Perl programs to link with dynamic libraries and call their functions on the fly.

The mechanics of passing arguments and returning values, unfortunately, depend on your machine, operating system, and compiler. Therefore, Makefile.PL checks the Perl configuration and may even run a test program before the module is built.

ExtUtils::DynaLib public constructor

The argument to new may be the file name of a shared library. Alternatively, a linker command-line argument (e.g., "-lc") may be specified. See DynaLoader(3) for details on how such arguments are mapped to file names.

Declaring a library routine

Before you can call a function in a shared library, you must specify its name, the return type, and the number and types of arguments it expects. This is handled by DeclareSub.

ExtUtils::DynaLib::DeclareSub can be used as either an object method or an ordinary sub. You can pass its arguments either in a list (what we call "positional parameters") or in a hash ("named parameters").

The simplest way to use DeclareSub is as a method with positional parameters. This form is illustrated in the first example above and both examples below. When used in this way, the first argument is a library function name, the second is the function return type, and the rest are function argument types.

C data types are specified using the codes used by Perl's pack and unpack operators. See perlfunc(1).

The arguments to DeclareSub are as follows:

name

The name of a function exported by $lib. This argument is ignored in the non-method forms of DeclareSub.

ptr

The address of the C function. This argument is required in the non-method forms of DeclareSub. Either it or the name must be specified in the method forms.

return

The return type of the function, encoded for use with the pack operator. Currently supported values are "" (void), "i" (int), "d" (double), "p" (a NUL-terminated character string), and "ptr" (any pointer type, usually equivalent to "i"; this is not a pack code because there isn't one for generic pointers that aren't dereferenced).

args

A list of the types of arguments expected by the function, specified using the notation of Perl's pack operator (see perlfunc(1)). For example, "i" means an integer, "d" means a double, "p" means a NUL-terminated string pointer.

Note: you probably don't want to use "c" or "s" here, since C normally converts the corresponding types (char and short) to int when passing them to a function. The ExtUtils::DynaLib package does not perform such conversions. Use "i" instead. Likewise, use "I" in place of "C" or "S", and "d" in place of "f". Stick with "i", "d", and "p" unless you know what you are doing.

decl

Allows you to specify a function's calling convention. This is possible only with a named-parameter form of DeclareSub. See below for information about the supported calling conventions.

libref

A library reference obtained from either DynaLoader::dl_load_file or the ExtUtils::DynaLib::LibRef method. You must use a named-parameter form of DeclareSub in order to specify this argument.

Calling a declared function

The returned value of DeclareSub is a code reference. Calling through it results in a call to the C function. See perlref(1) for how to call subs using code references.

Using callback routines

Some C functions expect a pointer to another C function as an argument. The library code that receives the pointer may use it to call an application function at a later time. Such functions are called callbacks.

This module allows you to use a Perl sub as a C callback, subject to certain restrictions. There is a hard-coded maximum number of callbacks that can be active at any given time. The default (4) may be changed by specifying CALLBACKS=number on the Makefile.PL command line.

A callback's argument and return types are specified using pack codes, as described above for library functions. Currently, the return value must be interpretable as type int or void, so the only valid codes are "i" and "". The first argument must be of type "i" (or possibly "p" on machines where integers are the same size as pointers). For argument positions beyond the first, the permissible types are "i", "p", and "d". These limitations are considered bugs to be fixed someday.

To enable a Perl sub to be used as a callback, you must construct an object of class ExtUtils::DynaLib::Callback. The syntax is

$cb_ref = new ExtUtils::DynaLib::Callback( \&some_sub,
                  $ret_type, @arg_types );

where $ret_type and @arg_types are the pack-style types of the function return value and arguments, respectively. Calling $cb_ref->Ptr() then returns a scalar whose integer value is the function address. C code that calls it will end up calling &some_sub.

EXAMPLES

This code loads and calls the math library function "sinh". It assumes that you have a dynamic version of the math library which will be found by DynaLoader::dl_findfile("-lm"). If this doesn't work, replace "-lm" with the name of your dynamic math library.

use ExtUtils::DynaLib;
$libm = new ExtUtils::DynaLib("-lm");
$sinh = $libm->DeclareSub("sinh", "d", "d");
print "The hyperbolic sine of 3 is ", &{$sinh}(3), "\n";
# The hyperbolic sine of 3 is 10.0178749274099

The following example uses the C library's "strncmp" to compare the first n characters of two strings:

use ExtUtils::DynaLib;
$libc = new ExtUtils::DynaLib("-lc");
$strncmp = $libc->DeclareSub("strncmp", "i", "p", "p", "i");
$string1 = "foobar";
$string2 = "foolish";
$result = &{$strncmp}($string1, $string2, 3);  # $result is 0
$result = &{$strncmp}($string1, $string2, 4);  # $result is -1

The files test.pl and README.win32 contain examples using callbacks.

CALLING CONVENTIONS

The problem

The hardest thing about writing this module is to accommodate the different calling conventions used by different compilers, operating systems, and CPU types.

"What's a calling convention?" you may be wondering. It is how compiler-generated C functions receive their arguments from and make their return values known to the code that calls them, at the level of machine instructions and registers. Each machine has a set of rules for this. Compilers and operating systems may use variations even on the same machine type. In some cases, it is necessary to support more than one calling convention on the same system.

"But that's all handled by the compiler!" you might object. True enough, if the calling code knows the signature of the called function at compile time. For example, consider this C code:

int foo(double bar, const char *baz);
...
int res;
res = foo(sqrt(2.0), "hi");

A compiler will generate specific instruction sequences to load the return value from sqrt() and a pointer to the string "hi" into whatever registers or memory locations foo() expects to receive them in, based on its calling convention and the types double and char *. Another specific instruction sequence stores the return value in the variable res.

But when you compile this module, it must be general enough to handle all sorts of function argument and return types.

"Why not use varargs/stdarg?" Most C compilers support a special set of macros that allow a function to receive a variable number of arguments of variable type. When the function receiving the arguments is compiled, it does not know how it will be called.

But the code that calls such a function does know, at compile time, how many and what type of arguments it is passing to the varargs function. There is no "reverse stdarg" standard for passing types to be determined at run time. You can't simply pass a va_list to a function unless that function is defined to receive a va_list. This module does use varargs/stdarg where appropriate, but the only appropriate place is in the callback support.

The solution

Having failed to find a magic bullet to spare us from the whims of system designers and compiler writers, we are forced to examine the calling conventions in common use and try to put together some "glue" code that stands a chance of being portable. Honestly, this work has just barely begun.

In writing glue code (that which allows code written in one language to call code in another), an important issue is reliability. If we don't get the convention just right, chances are we will get a core dump (protection fault or illegal instruction). To write really solid Perl-to-C glue, we would have to use assembly language and have detailed knowledge of each calling convention. Compiler source code can be helpful in this regard, and if your compiler can output assembly code, that helps, too.

However, this is Perl, Perl is meant to be ported, and assembly language is not portable. As of today, this module contains no assembly, though this may change in the future.

By avoiding the use of assembly, we lose some reliability and flexibility. By loss of reliability, I mean we can expect crashes, especially on untested platforms. Lost flexibility means having restrictions on what parameter types and return types are allowed.

The code for all conventions other than hack30 (described below) relies on the C alloca() function. Unfortunately, alloca() itself is not standard, so its use introduces new portability concerns. For cdecl, the most general convention, Makefile.PL creates and runs a test program to try to ferret out any compiler peculiarities regarding alloca(). If the test program fails, the default choice becomes hack30.

Supported conventions

ExtUtils::DynaLib currently supports the parameter-passing conventions listed below. The module can be compiled with support for one or more of them by specifying (for example) DECL=cdecl on Makefile.PL's command-line. If none are given, Makefile.PL will try to choose based on your Perl configuration and/or the results of running a test program.

At run time, a calling convention may be specified using a named-parameter form of DeclareSub (described above), or a default may be used. The first DECL=... supplied to Makefile.PL will be the default convention.

Note that the convention must match that of the function in the dynamic library, otherwise crashes are likely to occur.

cdecl

All arguments are placed on the stack in reverse order from how the function is invoked. This seems to be the default for Intel-based machines and possibly others.

sparc

The first 6 machine words of arguments are cast to an array of six ints. The remaining args (and possibly piece of an arg) are placed on the stack. Then the C function is called as if it expected six integer arguments. On a Sparc, the six "pseudo-arguments" are passed in special registers.

hack30

This is not really a calling convention, it's just some C code that will successfully call a function most of the time on most systems tested so far. All arguments are copied into an array of 6 integers (or 30 if 6 is not enough). The function is called as if it expected 6 (or 30) integer arguments.

You will run into problems if the C function either (1) takes more arguments than can fit in the integer array, (2) takes some non-integer arguments on a system that passes them differently from ints (but cdecl currently has the same flaw), or (3) cares if it is passed extra arguments. (This appears to crash certain Win32 functions including RegisterClassA(), which is used in the demo program in the README.win32 file.)

Because of these problems, the use of hack30 is recommended only as a quick fix until your system's calling convention is supported.

BUGS

Several unresolved issues surround this module.

Portability

The "glue" code that allows Perl values to be passed as arguments to C functions is architecture-dependent. This is because the author knows of no standard means of determining a system's parameter-passing conventions or passing arguments to a C function whose signature is not known at compile time.

Although some effort is made in Makefile.PL to find out how parameters are passed in C, this applies only to the integer type (Perl's I32, to be precise; see perlguts(1)). Functions that recieve or return type double, for example, will not work on systems that use floating-point registers for this purpose. To fix this will require changes to the C and XS code, as well as DynaLib.pm, though probably not to the documented public interface.

Robustness

Usually, Perl programs run under the control of the Perl interpreter. Perl is extremely stable and can almost guarantee an environment free of the problems of C, such as bad pointers causing memory access violations. Some Perl modules use a Perl feature called "XSubs" to call C code directly from a Perl program. In such cases, a crash may occur if the C or XS code is faulty. However, once the XS module has been sufficiently debugged, one can be reasonably sure that it will work right.

C code called through this module is devoid of such protection. Since the association between Perl and C is made at run time, errors due to incompatible library interfaces or incorrect assumptions have a much greater chance of causing a crash than with either static or XS code.

Security

This section is incomplete. I don't know what the security implications of this module may be. Use at your own risk.

Deallocation of Resources

To maximize portability, this module uses the DynaLoader(3) interface to shared library linking. DynaLoader's main purpose is to support XS modules, which are loaded once by a program and not (to my knowledge) unloaded. It would be nice to be able to free the libraries loaded by this module when they are no longer needed. This is impossible, since DynaLoader currently provides no means to do so.

Literal and temporary strings

Until Perl 5.00402, it was impossible to pass a string literal as a pointer-to-nul-terminated-string argument of a C function. For example, the following statement (incorrectly) produced the error "Modification of a read-only value attempted":

$strncmp->("foo", "bar", 3);

To work around this problem, one must assign the value to a variable and pass the variable in its place, as in

$strncmp->($dummy1 = "foo", $dummy2 = "bar", 3);

Callbacks

The Callback code is non-reentrant. And it assumes that pointers are packable as "I". Callbacks can mess up the message printed by die in the presence of nested evals.

Miscellaneous

There are far too many restrictions on what C data types may be used. Using argument types with size not a multiple of the machine word size may have nasty results. The techniques used to pass values to and from C functions are all rather hackish and not officially sanctioned. Assembly would be more robust, if less portable.

TODO

Fiddle with autoloading so we don't have to call DeclareSub all the time. Mangle C++ symbol names. Get Perl to understand C header files.

LICENSE

Copyright (c) 1997 by John Tobey. This package is distributed under the same license as Perl itself. There is no expressed or implied warranty, since it is free software. See the file README in the top level Perl source directory for details. The Perl source may be found at

http://www.perl.com/CPAN/src

AUTHOR

John Tobey, jtobey@user1.channel1.com

SEE ALSO

perl(1), perlfunc(1) (for pack), perlref(1), DynaLoader(3), perlxs(1), perlcall(1).