Did your installation of dotnetnuke/vs2008 work properly before installing Smith Cart?
1) There is a known problem with long path names and dnn. Today there is an unknown problem with www.dotnetnuke.com so I can't offer a reference. Try moving the web app to c:\dnn to see if things change.
2) It is not recommended to add AJAX to the \bin directory, it should be installed in the GAC (Global Assembly Cache) to prevent headaches from multiple AJAX versions.
v3.0.20229 was for .NET 3.5 & VS2008
v3.0.20820 was for .NET 3.5 SP1 & VS2008 SP1
v3.0.30930 shipped with Smith Cart v3.8.0
http://ajaxcontroltoolkit.codeplex.co...
You may want to rename the file to AjaxControlToolkit.dll.exclude and try browsing your site again.