When analyzing assembly code, be sure to pay attention to a routine’s arguments when the return is not apparent. For example, many times, returns are placed into eax and then eax is subsequently manipulated or read within a few instructions after a function call like this for example:

Here we see that immediately after LoadLibraryA is called, the value in eax is placed into edi and then edi is compared with the value in ebx, which in this case is 0. If the value returned by the function is indeed 0, then we jump to loc_405F28, otherwise we don’t take the jump and we continue down. It’s important to note that according to MSDN for LoadLibraryA¹,

If the function succeeds, the return value is a handle to the module.

Meaning that if the library is loaded successfully, this jump would not be taken and eax would contain the handle (identification number) to the newly loaded user32.dll library for future reference in the program. This is a pretty standard usage of a return seen in eax. However, in contrast, the below example can be tricky at first:

Here, the main purpose of GetModuleFilenameA is to of course get the module’s filename. However, the return value of this function is not the module’s filename. Instead, a pointer to a pre-declared buffer (lpFilename) must be supplied as an argument to the function, as is the case above where “offset Filename” is pushed just prior to the function call. This is of course standard practice C programming, however, when glossing over assembly code in a quick fashion, it can be easy to forget or miss the return values for these types of functions which place them into buffers passed in by pointers. In this example, GetModuleFileNameA modifies the memory pointed to by offset Filename, which is then placed into “ebp+arg_4” which is a memory location designated as an argument to the calling get_file_name function that we are in.

These arguments that are intended to be used to return data are labeled as OUT in MSDN:

It’s important to also remember that the function often still has a “return value” which is separate from the value placed into our “out” buffer and can be handled appropriately. In the case of GetModuleFileName, the return is as follows:

If the function succeeds, the return value is the length of the string that is copied to the buffer, in characters, not including the terminating null character. If the buffer is too small to hold the module name, the string is truncated to nSize characters including the terminating null character, the function returns nSize, and the function sets the last error to ERROR_INSUFFICIENT_BUFFER.

A good rule of thumb is if an expected return doesn’t appear to be in eax, check the arguments… And know your function arguments in general. In malware analysis, many of these Windows and Ntdll API calls are repeat offenders so this shouldn’t be too difficult. Til next time!