arrrggg -- strlcat

So, I am using strlcat and strlcpy in the project I'm working on. We had to implement our own version of these for Linux since it does not come with them. Well today I had a frustrating day of writing unit tests for this code. I developed the tests in cygwin as I wanted to validate that the native implementation and our implementation work the same. However, I find that cygwin doesn't work according to the available doc. So I went home to try it on my Mac and Linux boxes. So what do I find...

Well lets start with the return values section of Apple's man page:

RETURN VALUES
The strlcpy() and strlcat() functions return the total length of the string they tried to create.

Sun even documents this in better detail. The return value is:
min(sz, strlen(dest)) + strlen(src)

Well that seems simple enough, so what are the results of the tests. The dest in my tests is a 5 character array. I will represent the contents of this as a string literal.

0 == strlcat( "", "", 0 ) yes it does

3 == strlcat( "", "bbb", 0 ) no it returns 0

2 == strlcat( "aa", "", 5 ) yes it does

2 == strlcat( "", "bb", 5 ) yes it does

4 == strlcat( "aa", "bb", 5 ) yes it does

5 == strlcat( "", "abcde", 5 ) no this returns 4

9 == strlcat( "aaaa", "abcde", 5 ) no this returns 4

? == strlcat( "aa", "abcd", 1 )
Well what should the last one return. The case of dest not having a NULL within size characters is an error condition. I think that this should return 5 (size + strlen(src) ). But, no matter what I think, it actually returns 0.

So of course this means that the example that they give in the man page does not work.

if ( strlcpy( pname, dir, sizeof( pname ) ) >= sizeof( pname ) )
goto toolong;

As we can see from my example above, strlcat( "aaaa", "abcde", 5) only returns 4. And last I checked 4 is less than 5.

On Monday I'll have to try this out on Solaris to see what it says.

And don't get me started on the topic of the routines not validating their input. This is supposed to be a standard library, it should not cause a core dump when it receives bad input. Would it be so terribly difficult to return 0 and set errno if you got a NULL pointer?

Parsing Socket Connections With Flex and Bison

I developed a parser for my my current project using Flex and Bison. The development work was done reading from stdin by redirecting files. However, the intent of the system was for it to be run by inetd. Since inetd maps the accepted socket to stdin and stdout for you I didn't really foresee any problems. For testing and demo purposes I wrote a simple network version that accepted a connection and mapped it to stdin and stdout like inetd. However, I found that my program would block every time that it tried to write to the socket.

I traced the problem down to Flex but never came up with a reason for why it was happening. While looking into the problem I found the Lemon parser. The documentation for the Lemon parser mentioned reading from a socket into a buffer and then parsing that buffer. This was the key to solving the problem. My solution may seem to be a little convoluted but seems to work well. So here it is.

First I defined two functions, one that reads from a file descriptor and one that writes. These functions have prototypes of:

int parse_read(int fd, char *buff, int size, int *read )
int parse_write(int fd, char *buff, int size)

I then wrote a parser initialization routine parser_init with a prototype of.

void parser_init( int infd, int outfd, PARSE_READ inFunc, PARSE_WRITE outFunc)

The parser_init function will save the input and output file descriptors, the input and output call back functions, and initialize the buffer. Once this is done it calls the flex function yy_scan_string. I did create two default functions that are used if NULL is passed for the callback parameters. But allowing for different input and output functions to be used makes things like unit testing much easier.

I then wrote a yywrap function in my Bison input file. The yywrap function, which is called when flex's input buffer is empty, calls the read callback function to get more data. If the callback returns 0, success, yywrap calls yy_scan_string again.

This probably does not make too much sense until you see it in action. So hopefully this will help to illustrate how all of the pieces come together, connfd is our connected socket and the default read and write routines will be used.

• The application calls parser_init( connfd, connfd, NULL, NULL )
• parser_init sets the input and output file descriptors
  
• parser_init sets the callbacks to the defaults
  
• parser_init initializes the buffer and calls yy_scan_string
• the application calls yyparse to start the parser
• flex finds that the input buffer is empty and calls yywrap
• yywrap calls the read callback.
• the read callback reads from the socket and populates the buffer.
• yywrap calls yy_scan_string
• yywrap returns 0 to have flex continue, if there was data available.

And that is the process, which actually seems to work quite well. Except for the small memory leak, but that is another post.

Fortune Cookie Says

Your principles mean more to you than any money or success.

Setting Environment Variables From Make Targets -- part II

I ran into a issue with the way that GNU make acts between cygwin and Solaris. In my previous post I used two instances of the same target, one to export the environment variable and one that executes the test. For example:

TST_CONFIG_MODULE := $(d)/test_config
...
check_config : export TST_DIR:= \
$(dir $(TST_CONFIG_MODULE) )
check_config : $(TST_CONFIG_MODULE)
$(TST_CONFIG_MODULE)

This is how it is documented in the GNU make manual and how it works under Cygwin. However, this fails on Solaris. The version of make installed on Solaris is only a couple of point releases different than the one on Cygwin so I wouldn't expect this to be an new feature. The immediate solution that I came up with is the same one that I used for the main check target.

check_config : $(TST_CONFIG_MODULE)
@for m in $(TST_CONFIG_MODULE) ; do \
export TST_DIR=`dirname $$m`; \
$$m;
done

Setting Environment Variables From Make Targets

In my current project I needed to pass some pass information to some of my automated unit test programs. I needed the path that the program is being executed from to be able to read test data. As I am using a unit test framework I didn't have access to the command line variables and didn't want to take the time to change the framework to support this. So I decided to define an environment variable with the path information for the program. The unit tests can be run in two different ways, which needed different mechanisms to define the environment variable. The makefiles are setup with rules to run a single set of tests (e.g. make check_config), or running all unit tests (e.g. make check).

To support running a single set of unit tests the make file defines two targets. One that exports the environment variable and one that executes the test. For example:

TST_CONFIG_MODULE := $(d)/test_config
...
check_config : export TST_DIR:= \
$(dir $(TST_CONFIG_MODULE) )
check_config : $(TST_CONFIG_MODULE)
$(TST_CONFIG_MODULE)

Where the TST_CONFIG_MODULE defines the test program. The first check_config target exports an environment variable TST_DIR that is the directory component of, $(d), of TST_CONFIG_MODULE. The make file has to be structured this way as $(d) will not be valid by the time the target is evaluated.

For the second case the makefiles add test modules to the macro CHECK_MODULES and the check target just executes each element of this macro. As this is executed in a shell the target becomes:

check : $(CHECK_MODULES)
@for m in $(CHECK_MODULES) ; do \
export TST_DIR=`dirname $$m`; \
$$m;
done

It is a bit of a kludge to use the dirname executable to get the directory path from the test program name. Doing this assumes that we have dirname installed and that it is on the path. Bash has a built in to do this, ${dirname file}, but there does not seem to be such a builtin in sh.

Setting a Macro to Wildcarded Files

I found a mistake in one of my macro definitions. The macro was to hold all of the public header files and was defined as:

HEADERS := $(d)/*.h

This only set HEADERS to src/include/*.h instead of setting it to the files. To get they desired result the macro was changed to:

HEADERS := $(wildcard $(d)/*.h)

Fortune Cookie Says

Hard words break no bones, fine words butter no parsnips.