If you are C/C++ developer that needs to create applications using contemporary Microsoft Visual C++ compilers that are linked to the MSVCRT.DLL only, then you should read this article. However if you are lucky and you are creating self contained applications which you totally control and which do not allow third party plug-ins then this article is probably of no interest to you.
I was intrigued by the fact that majority of Microsoft products are linked only to MSVCRT.DLL and that those CRT’s are not the same as one used or produced by Visual Studio 6.0 compiler and CRT sources. There are few articles you can find on the Internet about the topic, but I didn’t found any of them actually helpful, so I decided to dig in and create something usable.
With Visual Studio .NET back in 2002 Microsoft introduced a new CRT DLL and all hell broke loose. Since then each new version of Visual Studio ships with a new CRT DLL making the applications or libraries bound to that particular DLL. Here is the table that shows those version
- MSVCRT70.DLL Visual Studio .NET
- MSVCRT71.DLL Visual Studio 2003
- MSVCRT80.DLL Visual Studio 2005
- MSVCRT90.DLL Visual Studio 2008
- MSVCRT100.DLL Visual Studio 2010
The major problem that arises when using multiple CRT versions is explained in this and this MSDN article.
As you can see, one thing that Microsoft tries to convince you is that MSVCRT.DLL is of no interest to you and that you should use the one that comes with Visual Studio, and even more that you should make sure that each and every component is build using exactly the same version of C compiler. Nice in theory but if it happens that you need to embed already produced binaries compiled by someone else then this is not an option. So if MSVCRT.DLL is of no interest to you how come that lots of Microsoft programs are bound to that DLL instead eating their own dog food? I suppose there is a bit truth in that old latin say “Quod licet Iovi, non licet bovi”
Few things that MSDN article doesn’t mention are inherent cross bounday objects like stdio objects. If you redirect your stdout results can be surprising.
DLL
#include
#include
__declspec(dllexport) void __stdcall dllhi(void)
{
printf("Hi from DLL\n");
}
Test program
#include
#include
__declspec(dllimport) void __stdcall dllhi(void);
void redirect(const char *fname)
{
/* Reassign "stdout" to fname */
freopen(fname, "w", stdout);
}
void sayhi(void)
{
printf("This will go to the test.out\n");
}
int main(int argc, char *argv[])
{
redirect("test.out");
sayhi();
dllhi();
return 0;
}
Compile DLL
cl /MD /DLL dll.c
Compile Test program
cl /MD tst.c dll.lib
If you compile both DLL and Test program with the same compiler everything works as expected. However if you compile tst.c with VS2003 and dll.c with VS2008 the application won’t even initialize complaining about Runtime initialization. Other way around works if you compile tst.c with VS2008 and dll.c with VS2003 but the message from DLL won’t be present in the output file. The reason is because each MSVCRTnn.DLL maintains its own copy of those system objects like stdio and stderr streams.
First problem is more easily handled although the error message box doesn’t say much about the reason of failure, but the second is more subtle. Everything seems to be working but you are loosing the data from the DLL that would go to redirected file. This means that each and every CRT stdio function is unsafe when crossing DLL boundaries even if you don’t pass them directly. Also almost all functions that deal with memory allocation are unsafe as well. This means that you cannot free memory that was allocated in a DLL. You cannot use Posix file descriptors as well. For example using an descriptor obtained by calling open() in DLL would result in access violation for any operation on that descriptor performed from the object file linked with different CRT. Just like for stdio and stderr each CRT maintains it’s own list of pseudo file handles. In worse case your data can end up in unexpected location if it happens that you call open() from different DLL’s.
So what happens if you link with system MSVCRT.DLL. Well it doesn’t solve the issues, but it won’t crash your application neither will your DLL (if that’s what you are building) won’t crash the host.
To fight the information loss across different CRT DLL boundaries one solution is to update the data in all CRT DLL’s currently loaded in process. This can be very complex as shown on PostgreSQL example. As you can see they try to load each and every possible MSVCRTnn.DLL and obtain a pointer to putenv() function and calling that function. However this still won’t work for a simple printf() with redirected stdio stream. Acutally I’m not even sure there is a solution for that, beside using the same CRT of course.
The only safe solution is to use the same CRT for your application and your application’s modules. This has some drawbacks of course, and the major one is that you simply must ensure that all dependencies are using the same compiler generation and thus linking to the same CRT DLL. Within Apache Httpd we were using good old Visual Studio 6.0 for producing official binaries, but recently there were some discussion to switch to Visual Studio 2010. One of the major reasons for that is because you just cannot download Visual Studio 6.0 any more, even if you’ve purchased MSDN subscription. It’s not maintained any more as well, and it cannot create 64 bit binaries so it’s logical to move forward to something more advanced.
There is however a solution to use the system MSVCRT.DLL without the need to ship the CRT redistributables which mandate using some sort of installer and Microsoft insist that this should be .msi. Solution is to use the Windows Driver Kit which is freely downloadable from MSDN and can be used for producing a binaries linked with MSVCRT.DLL. Driver kit is meant to be used for producing OS drivers and since this is very sensitive part of your system it must be very stable and error prone. You might ask yourself like I did: OK if drivers which first premise is stability must be compiled to MSVCRT.DLL then why everything else must be compiled with MSVCRTnn.DLL. Weird statement and I really have no clue why Microsoft decided to go that way. At least I didn’t find any technological reason for such a move. Well, I understand that MSVCRT.DLL in Windows XP is different the in Windows 7, but that could at least make sure that higher MSVCRTnn.DLL use data structures from MSVCRT.DLL each of them maintaining its own copy or parallel implementation.
Anyhow, I decided to go with the Windows DDK path. However this created a whole bunch of problems. One of the the first one I faced was that msvcrt.lib from DDK doesn’t export few symbols for functions or data that is in the MSVCRT.DLL like getpid(), _environ and _wenviron. I would like to think that this is just a bug rather then intentional. This can be solved pretty easy by creating a library which will export those symbols, but the drawback is that you must link to that library. At the end this means you must modify you make files to include that thunk library. Create a file named msvcrt_compat.def with the following content:
EXPORTS
_getpid=msvcrt._getpid
_environ=msvcrt._environ
_wenviron=msvcrt._wenviron
Then create a thunk library with those missing symbols using
lib /NOLOGO /NODEFAULTLIB /DEF:msvcrt_compat.def /MACHINE:X86 /NAME:msvcrt.dll /OUT:msvcrt_compat.lib
All you have to do now is to link your existing application with msvcrt_compat..lib and msvcrt_win2003.obj or msvcrt_winxp.obj from DDK. I have made a toolkit as part of my next pet project that does that automatically for you. It creates a custom Visual Studio compilation using Windows Driver Kit and Windows Software Development Kit. It’s currently hosted on sourceforge and it contains some additional stuff which you are probably not interested in, like optionally installing Cygwin and Perl. Anyhow for those that just need a compiler they will need the following.
- Windows Server 2003 R2 Platform SDK
Make complete install inside default directory
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2 - Windows Software Development Kit for Windows 7
Make complete install inside default directory.
C:\Program Files\Microsoft SDKs\Windows\v7.0\
It will also install a subset of Visual Studio 2008 (9.0) at
C:\Program Files\Microsoft Visual Studio 9.0 - Windows Driver Kit version 7.1.0
This is a DDK version for Windows 7 and Windows Server 2008r2 There are two versions of those at MSDN, so make sure to download version 7.1.0 or later. It installs in: c:\WinDDK\7600.16385.1 - Toolkit for making a custom MSVC distribution
svn co https://myomake.svn.sourceforge.net/svnroot/myomake/trunk/misc/tools/windows/cmsc
Make sure you read the README.txt files
If you install all the mentioned packages and checkout the toolkit from MyoMake sourceforge project, open a command prompt and cd to tools directory. Execute cmsc_compile.bat script and it will create a workable compiler distribution for you. Note that you are probably not allowed to redistribute that to someone else according to the Microsoft license terms (don’t say you haven’t be warned). You can call cmsc_makedist.bat which will create a .zip file which you can use for backup purposes or for your dev box. I have created a setup script for command line compilation (setenv.bat) which when called set’s up the correct compiler in a similar way SetEnv.cmd does with Platform SDK. It also sets EXTRA_LIBS environment variable which you can use to add required thunk at link time as well as BUILD_CPU with selected architecture (i386, amd64 or ia64). After calling setenv.bat /x86 the upper examples can be compiled
cl /MD /DLL dll.c %EXTRA_LIBS%
cl /MD tst.c dll.lib %EXTRA_LIBS%
Note that this is not a users manual for that custom compilation script. I’m not even sure it will survive the next DDK version, but who knows. Microsoft might surprise us. Anyhow I’d like to hear your experiences with the subject. When making a comment make sure you read the rules.
March 4, 2011 at 13:39
Hi,
This is a very useful article and tools but I am having some minor conceptual issues. First, what is the purpose of the Windows Server 2003 R2 Platform SDK and why do you need to copy stuff out of it? I’m an MSDN subscriber and I can’t find it anywhere under the subscriber downloads.
Second, since the issue we’re all trying to solve is to use msvcrt.dll instead of the version used by Visual Studio (in my case VS2010), why exactly is it that you need to create this full build environment from the various components as your batch files does? Surely after creating he extra thunk library once, I can just copy that into my project directory, for example, and link against it? I would like to take advantage of the more modern compiler in VS2010 and I need to build a fair number of DLL’s, and I would rather have them link against msvcrt.dll, but I would also rather not leave the VS2010 environment completely. Is this possible?
Thank you very much for your time and this information.
March 4, 2011 at 15:27
The purpose of Win 2K3 Platform SDK is to provide Windows SDK includes and libraries. Might work with some contemporary SDK’s but didn’t try. If you have MSDN you can download it by clicking on “Operating Systems -> Windows 2003 R2″. There is a “Platform SDK for Windows Server 2003 R2 – March 2006 Edition (English)”
Here is the full link
March 4, 2011 at 15:32
Thanks, I found it. For your information I am going to try the batch files with “Microsoft SDK For Windows 7 and Windows Server 2008 R2″ just to see how it plays out. Any insight or answer to my second question?
March 4, 2011 at 15:51
Well it’s not just library. What counts is the compiler itself, so you have to use the compiler from DDK. I don’t use IDE, but I suppose you can adjust the Directories for exe, include and library and give it a try.
March 15, 2011 at 15:04
It seems to work fine, thanks. Only a couple of notes. First, the patch files provided (crtdefs.patch and delayimp.patch) don’t seem to work unless converted to DOS text format (CR+LF). Second, the setenv.bat contains a typo (“tols” instead of “tools”) and, IMHO, the directory “%VSRootDir%\perl\bin” shouldn’t be added to the PATH if Perl is not installed. But after fixing these issues, everything works OK for me.
March 21, 2011 at 19:00
Two more symbols for you to add to msvcrt_compat.def, without which glib can’t compile statically:
__wgetmainargs=msvcrt.__wgetmainargs
__getmainargs=msvcrt.__getmainargs
March 21, 2011 at 20:52
It seems those exports are not public and are used by CRT internally. However since I confirmed they are present in XP and Win7 system msvcrt.dll’s, committed. Thanks!
June 28, 2011 at 14:30
Hello,
I tried to apply what you explained. However, I still have the same message when I launch the exe file. What I did:
- I got all the different packages you mentionned and installed it.
- # cmsc15_compile.bat
I got this error message (several times) :
“Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354
- # msvcrt_compat.bat
- add msvcrt_compat.lib to the properties of the VS solution
- and compile.
Did I miss something?
In advance, thank you for your help.
June 28, 2011 at 17:07
All you have to do is to install the required Windows SDK’s and DDK and then
1. Open cmd.exe in a folder without spaces
2. svn co https://myomake.svn.sourceforge.net/svnroot/myomake/trunk/misc/tools/windows/cmsc
3. cd cmsc\tools
4. cmsc_compile
This procedure should work without any problems. At least I don’t see that patch is failing.
June 28, 2011 at 17:16
Also make sure to use the true win32 svn client. The patch files have ‘native’ line endings property for svn:eol-style, so if you use cygwin svn they will get checkout with unix line endings. Alternative if using cygwin is to call ‘unix2dos tools/crt*.patch’
July 2, 2011 at 15:53
I tried to build libxml2 but received __imp__vsnprintf not found error.
I added the symbol
vsnprintf=msvcrt.vsnprintf
in msvcrt_compat.def, seems I can compile successfully now.
July 2, 2011 at 22:13
At least Windows XP msvcrt.dll doesn’t have vsnprintf exported, only _vsnprintf.
They have the same API so
#define vsnprintf _vsnprintf
should work thought
I’ll see how other ANSI functions for which Microsoft (that was always puzzling me) decided to have underscore prefix are made.
Anyhow, think this should actually be
vsnprintf=msvcrt._vsnprintf
July 4, 2011 at 09:14
Ok, thank you very much.
The unix2dos tools/crt*.patch command solved the “Assertion failed: hunk, file ../patch-2.5.9-src/patch.c, line 354″ problem.
Now, if I link with msvcrt_compat.lib and msvcrt_compat.obj, I have several compilation error messages, such as:
>msvcrt_compat.obj : error LNK2001: external symbol _strcpy_s
>msvcrt_compat.obj : error LNK2001: external symbol _strnlen
If I compile only with msvcrt_compat.lib, I have the same _except_handler4_common error while on execution.
July 4, 2011 at 10:12
Hmm, You will usually see LNK2001 for _strnlen if you are compiling without /MD switch. By default cl.exe uses /SD model, so make sure you compile with /MD
July 4, 2011 at 12:10
I have both 2001 and 2019 error types. Actually I do compile with /MD. Tying to find out what’s going on…
August 9, 2011 at 11:27
Just FYI, the MSVCRT in Windows is basically just the MSVCRnnn.DLL from the most recent compiler when that Windows version was shipped. The idea is that MSVCRT is forward compatible (if you compile against MSVCRT in WinXP then it should work in Vista/7 but if you compile against MSVCRT in Win7 then it may or may not work on Vista/XP).
As for why MS doesn’t want people using MSVCRT directly, due to the above problem it is easier to maintain compatibility by keeping the same runtime DLL across all Windows versions (in standalone apps) and newer compilers may assume the presence of certain runtime functions (it’s perfectly legit [standard-wise] for the compiler to insert calls to the runtime library wherever it likes which gets fun if that function is only in the newest MSVCRnnn). In the end, it makes things easier for MS’ compiler team but worse for everyone else.
Given the above, the reason the DDK allows linking to MSVCRT is because drivers are supposedly only built for a single “platform” (version of Windows in MS speak) which basically means that the driver (and supporting user software) is compiled for WinXP or 2003 or Vista or 7 and if it works on any other version then you won a dice roll. OTOH, the forward compatibility guarantee [minus deprecated functions] also seems to apply to the kernel so it’s a little less risky than MS paints it but they clearly don’t want people getting comfortable with it. [Naturally, MS does use MSVCRT the exact way you've outlined since things like control panel applets and device manager configuration pages are DLLs hosted by a system provided EXE, having the DDK link MSVCRT over MSVCRnnn solves the runtime problem for Windows components]
This leaves us to speculate what exactly we are supposed to do, in MS’ eyes, to get around the fact that file handles and such are implemented in the C Runtime rather than the system. The obvious answer is that they want you to use WriteConsole and FormatString instead of printf, CreateFile instead of fopen, etc then pass around HANDLEs instead of FILE*s. Kernel32/User32/etc are pretty much the only universal DLLs in the system which make them the only non-hack way out of this hole. This is probably the best reason to explain why they won’t fix it (If they just made MSVCRT backwards compatible then everyone could link to it, have the same version across all apps and everything would just work — quite telling that this is basically the ONLY major part of Win32 which ISN’T backwards compatible)
Reply
August 9, 2011 at 13:36
With DDK they provide a thunk for the missing stuff in the OS CRT (e.g XP). Linking to msvcrt_winxp.obj or msvcrt_win2003.obj is required to inject the code into the produced dll or exe for dealing with mostly exception handling. AFAICT this is the only thing missing from the system MSVCRT.dll and is required by the latest compilers. And I wouldn’t name this “plug and pray” like with drivers, cause you know exactly what is going on (well, sort of, because Microsoft has never released the exception handling code). Given that, I find using DDK compiler pretty safe for producing binaries for multiple platforms because the linked code only inserts the exception code and some obscure Microsoft additions like strcpy_s which are not found in earlier system CRT’s.
Reply