main
1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
2<html>
3 <head>
4 <!-- $Id: NCoverFAQ.html 247 2007-04-26 15:17:46Z peterwald $ -->
5 <title></title>
6 <meta content="Microsoft Visual Studio .NET 7.1" name="GENERATOR">
7 <meta content="http://schemas.microsoft.com/intellisense/ie5" name="vs_targetSchema">
8 <style> body { font-size: 10pt; font-family: Verdana; }
9 p.title { font-size: 20pt; font-weight: bold; }
10 .subtitle { color: maroon; }
11 p.question { font-weight: bold; }
12 pre { font-size: 10pt; font-family: Courier; }
13 pre.usage { background-color: #F0F0F0; }
14 .quote { background-color: #F0F0F0; margin-left: 36pt;}
15 .method { color: maroon; font-size: 10pt; font-weight: bold; }
16 .hdrcell { background-color: #DDEEFF; font-size: 10pt; }
17 .datacell { background-color: #FFFFEE; text-align: right; font-size: 10pt; }
18 .hldatacell { background-color: #FFCCCC; text-align: right; font-size: 10pt; }
19 .box { border: 1px solid; padding: 10px; }
20 </style>
21 </head>
22 <body>
23 <P class="title">NCover FAQ</P>
24 <P>If you have questions that this document does not address, contact <A href="mailto:peter@waldschmidt.com">
25 Peter Waldschmidt</A> or try the <A href="http://ncover.org/">NCover Forums</A>.</P>
26 <P class="question">1. What is code coverage analysis?</P>
27 <P class="answer">A code coverage analyzer monitors your code at runtime and
28 records information about which lines of code were executed. NCover shows each
29 sequence point in your application along with the number of times that point
30 was executed. Sequence points are generated by the compiler and stored in the
31 debug information (.pdb) files. A sequence point basically corresponds to a
32 single program statement (often a line of code) in your high-level language.</P>
33 <P class="question">2. Why would I want to do code coverage analysis?</P>
34 <P class="answer">Unit test suites are often used as a quality tool during the
35 development process to keep the codebase stable as it changes and expands.
36 Tools such as <A href="http://nunit.org/">NUnit</A> are often used to run and
37 report on the test suites. However, when implementing unit testing in your
38 build process, you have no way of knowing how much of your code the unit tests
39 are actually testing. This is where code coverage comes in. You can run NUnit
40 within NCover and use the code coverage report to determine which code was not
41 tested by that particular test suite.</P>
42 <P class="question">3. What versions of the CLR does NCover support?</P>
43 <P class="answer">
44 NCover 1.5.x requires the .NET framework version 2.0.50727 to be installed; however,
45 the application being profiled can be written against any shipping version of the
46 framework. NCover
47 has been tested profiling coverage of .NET 2.0, .NET 1.1 and .NET 1.0 applications.</P>
48 <P class="question">4. Which version of NCover should I install?</p>
49 <P class="answer">
50 If you have the .NET 2.0 framework installed on your machine then you should use
51 the latest NCover version available. NCover as of version 1.5 can profile .NET 2.0, 1.1 and 1.0 applications.</p>
52 <p class="answer">
53 For development teams who do not have the .NET framework 2.0 installed but do have
54 the .NET framework version 1.1.4322, you can
55 try NCover 1.3.3. Note however that this version is no longer supported as
56 it has a number of known issues and limitations.</p>
57 <P class="question">5. What is the command line syntax for NCover?</P>
58 <P class="answer">Here is the usage info from the NCover command line (for NCover versions from 1.5.6
59 only):</P>
60 <pre class="usage">NCover.Console [<command line> [<command args>]]
61 [//svc <service name>]
62 [//iis]
63 [//a <assembly list>]
64 [//w <working directory>]
65 [//ea <exclusion list>]
66 [//reg]
67 [//x <xml output file>]
68 [//s [<settings file>]] [//r [<settings file>]]
69 [//v] [//q]
70 [//l <log file>]
71
72//svc For profiling windows services
73//iis For profiling web applications
74
75//a List of assemblies to profile separated by semi-colons i.e. "MyAssembly1;MyAssembly2". Do not include paths or suffixes.
76//w Working directory for profiled application
77//ea List of attributes marking classes or methods to exclude from coverage
78
79//reg Register profiler temporarily for user. (helps with xcopy deployment)
80//x Specify coverage output file. (default: coverage.xml).
81//pm Specify name of process to profile (i.e. myapp.exe)
82
83//s Save settings to a file (defaults: NCover.Settings)
84//r Use settings file, overriding other settings (default: NCover.Settings)
85
86//l Specify profiler log file (default: coverage.log).
87//q No logging (quiet)
88//v Enable verbose logging (show instrumented code)
89 </pre>
90 <UL>
91 <LI><command line> - This argument specifies the command-line of the .NET application
92 you want to analyze.
93 Any command line arguments not starting with // will be passed
94 through to that application. NCover will profile the running application until it has exited. See below for examples.<li>//svc - This option is an alternative to the <command line>
95 for profiling windows services, which cannot be run directly as executables. NCover
96 will start the service (stopping it first if already running) and profile coverage
97 until the windows service is stopped.</li>
98 <li>//iis - This option is an alternative to the <command line> for profiling
99 web applications. NCover will start the IISAdmin and W3C
100 services (stopping first if currently running) and profile coverage until the IISAdmin
101 service is stopped.<br />
102 </li>
103 <li>//a - This command-line argument specifies the assemblies that you want to analyze.
104 NCover can only analyze assemblies that have .pdb files included with them. If
105 you do not specify the //a argument, NCover will attempt to analyze every loaded
106 assembly that has debug information available. Note that the assembly name arguments are
107 the module name within the assembly, not the physical file name. e.g. "MyAssembly"
108 rather than "MyAssembly.dll".<li>//w - If the application being profiled requires the
109 working directory to be set to something other than the current directory you are
110 executing the command line from then you can override it with this argument.</li>
111 <li>//ea - You can choose to exclude classes and methods
112 from coverage statistics by defining .NET attribute(s) and applying it to the affected
113 code. When using this argument you must specify the full type namespace of these
114 attribute(s) separated by semi-colons. See below for an example.<br />
115 </li>
116 <li>//reg - NCover requires a COM registration of the CoverLib.dll assembly containing
117 the profiler, which is performed automatically by the default .msi installation.
118 If you require an xcopy style deployment of NCover like many other .NET tools, then
119 you can use this argument which will temporarily register the profiler while performing
120 coverage. This feature was added in NCover 1.5.6.</li>
121 <li>//x - The output of NCover is an xml file (example below). Use this argument to
122 specify an alternate filename to "coverage.xml" in the current directory.<br />
123 </li>
124 <li>//pm - This setting tells NCover to ignore processes that don't have the specified process module name.
125 This is the name of the executable (i.e. myapp.exe). This setting is useful in cases, where your NCover
126 command spawns a series of child processes. Using this setting will help NCover determine which process to profile.
127 </li>
128 <li>//s - You may find it more convenient to use a settings file rather than specifying
129 a long list of command line arguments for running NCover. If you get the NCover
130 command line working as you would like it and then use the //s argument it will
131 save the required arguments as an xml file that can then be used by the //r argument
132 below.</li>
133 <li>//r - For use when you have used //s to construct an NCover settings file containing
134 your command line arguments. e.g. "ncover.console.exe //r NCover.Settings"<br />
135 </li>
136 <li>//l - The coverage log file can provide an insight if the desired coverage output
137 is not obtained. Useful information you may find to assist you includes which assemblies
138 were loaded by NCover, their file paths and which of those it found the .pdb build
139 symbols for. Use this argument to specify an alternative log file name or location
140 to coverage.log in the current directory.</li>
141 <li>//q - Suppresses writing the coverage.log file.</li>
142 <li>//v - This command-line argument makes the profiler emit all the original IL and
143 modified IL instructions to the coverage log. This is useful for debugging
144 purposes. Beware that this can make your coverage log file very large!
145 </li>
146 </UL>
147 <P class="question">6. Does NCover required a special compilation step for my code?</P>
148 <P class="answer">No. Some code coverage tools change your source code and force
149 you to recompile it into a special build. NCover is designed to work
150 on shipping code. NCover uses the .NET Framework profiling API to monitor
151 your code. It does require build symbols, but can be run on release code
152 without any modifications.</P>
153 <P class="question">7. How does NCover work?</P>
154 <P class="answer">NCover uses the .NET Framework profiler API to monitor an
155 application's execution. When a method is loaded by the CLR, NCover retrieves
156 the IL and replaces it with instrumented IL code. NCover does not change
157 your original IL code, it simply inserts new code to update a visit
158 counter at each sequence point. Upon
159 request, (usually after the .NET process has shut down) the profiler outputs statistics
160 to the coverage file.
161 </P>
162 <P class="question">
163 8. What is the output of NCover?</P>
164 <P class="answer">NCover generally writes out three files after analysis
165 completes.
166 <ul>
167 <li>
168 Coverage.log - This file is a log of the events and messages from the profiler
169 during the analysis process. Most of the time, error messages are recorded in
170 this log. If you enable verbose logging, the coverage log will contain
171 disassembly of the original and instrumented IL code. Verbose logging is not recommended for
172 normal use.<li>
173 Coverage.xml - This file is the analysis output of NCover. You can see an
174 example of the output below.
175 <LI>
176 Coverage.xsl - This file is a simple XML transformation that makes the XML
177 output easily readable.
178 </LI>
179 </ul>
180 <span class="subtitle">Example XML output</span>
181 <div class="box"><pre><method class="NCoverTest.ClassLoaded" name="HasDeadCode">
182 <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs"
183 column="13" line="48" endcolumn="58" endline="48" visitcount="1" />
184 <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs"
185 column="13" line="49" endcolumn="22" endline="49" visitcount="1" />
186 <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs"
187 column="17" line="50" endcolumn="24" endline="50" visitcount="1" />
188 <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs"
189 column="13" line="51" endcolumn="48" endline="51" visitcount="0" />
190 <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs"
191 column="9" line="52" endcolumn="10" endline="52" visitcount="0" />
192</method></pre>
193 </div>
194 <p></p>
195 <span class="subtitle">Example transformed output</span>
196 <div class="box">
197 <DIV class="method">NCoverTest.ClassLoaded.HasDeadCode</DIV>
198 <TABLE id="Table1" borderColor="black" cellSpacing="0" cellPadding="3" border="1">
199 <TBODY>
200 <TR>
201 <TD class="hdrcell">Visit Count</TD>
202 <TD class="hdrcell">Line</TD>
203 <TD class="hdrcell">Column</TD>
204 <TD class="hdrcell">End Line</TD>
205 <TD class="hdrcell">End Column</TD>
206 <TD class="hdrcell">Document</TD>
207 </TR>
208 <TR>
209 <TD class="datacell">1</TD>
210 <TD class="datacell">48</TD>
211 <TD class="datacell">13</TD>
212 <TD class="datacell">48</TD>
213 <TD class="datacell">58</TD>
214 <TD class="datacell">C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs</TD>
215 </TR>
216 <TR>
217 <TD class="datacell">1</TD>
218 <TD class="datacell">49</TD>
219 <TD class="datacell">13</TD>
220 <TD class="datacell">49</TD>
221 <TD class="datacell">22</TD>
222 <TD class="datacell">C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs</TD>
223 </TR>
224 <TR>
225 <TD class="datacell">1</TD>
226 <TD class="datacell">50</TD>
227 <TD class="datacell">17</TD>
228 <TD class="datacell">50</TD>
229 <TD class="datacell">24</TD>
230 <TD class="datacell">C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs</TD>
231 </TR>
232 <TR>
233 <TD class="hldatacell">0</TD>
234 <TD class="datacell">51</TD>
235 <TD class="datacell">13</TD>
236 <TD class="datacell">51</TD>
237 <TD class="datacell">48</TD>
238 <TD class="datacell">C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs</TD>
239 </TR>
240 <TR>
241 <TD class="hldatacell">0</TD>
242 <TD class="datacell">52</TD>
243 <TD class="datacell">9</TD>
244 <TD class="datacell">52</TD>
245 <TD class="datacell">10</TD>
246 <TD class="datacell">C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs</TD>
247 </TR>
248 </TBODY>
249 </TABLE>
250 </div>
251 <P>Suggested usages of the coverage.xml output are to display it in the <a href="http://ncoverexplorer.org/">
252 NCoverExplorer</a> gui with the source
253 code highlighted, to generate html reports, or to include it in your continuous build server reports such as CruiseControl.Net.
254 For more information on these options see below in the FAQ.</P>
255 <P></P>
256 <P class="question">
257 9. How do I use coverage exclusions?</P>
258 <p>
259 First you should define an attribute to markup your excluded code with. You will
260 likely want to put this in a common assembly to make it reusable, or indeed within
261 a "CommonAssemblyInfo.cs" that you include in all your application assemblies.</p>
262 <P></P>
263 <pre class="usage">namespace MyNamespace {
264 class CoverageExcludeAttribute : System.Attribute { }
265}</pre>
266 <p>
267 Apply the attribute to the C# classes and/or methods you wish to mark as excluded
268 from code coverage statistics:</p>
269 <P></P>
270 <pre class="usage"> [CoverageExclude]
271 private void SomeMethodToExclude() {} </pre>
272 <p>
273 Finally, ensure you pass the full qualified attribute information in the NCover command line:</p>
274 <P></P>
275 <pre class="usage"> NCover.Console MyApplication.exe //ea MyNamespace.CoverageExcludeAttribute </pre>
276 <p>
277 Note that if you are using the <a href="http://testdriven.net/">TestDriven.Net</a>
278 VS.Net add-in to "Test with Coverage" it will automatically
279 pass through "//ea CoverageExcludeAttribute"
280 which you should define without a namespace like above. For further information refer to this
281 <a href="http://weblogs.asp.net/nunitaddin/archive/2006/10/04/CoverageExclude.aspx">
282 blog entry</a>.</p>
283 <P class="question">
284 10. Examples</P>
285 <p>
286 Coverage while running a simple executable until it exits:</p>
287 <P></P>
288 <pre class="usage"> NCover.Console MyApplication.exe</pre>
289 <p>
290 Coverage while running all the unit tests in an assembly using NUnit, profiling
291 all loaded assemblies with .pdb build symbols:</p>
292 <P></P>
293 <pre class="usage"> NCover.Console nunit-console.exe MyApplication.Tests.dll</pre>
294 <p>
295 Coverage of only a subset of loaded assemblies while running unit tests:</p>
296 <P></P>
297 <pre class="usage"> NCover.Console nunit-console.exe MyApplication.Tests.dll //a MyApplication.Core;MyApplication.Utilities</pre>
298 <p>
299 Coverage of a windows service. Stop the service to generate the coverage output:</p>
300 <P></P>
301 <pre class="usage"> NCover.Console //svc MyServiceName</pre>
302 <p>
303 Coverage of an ASP.Net application. Stop the IIS service to generate the coverage
304 output:</p>
305 <P></P>
306 <pre class="usage"> NCover.Console //iis</pre>
307 <P class="question">
308 11. Where can I get help or support?</P>
309 <P class="answer">
310 Your best approach is to browse the <a href="http://ncover.org/site/forums/default.aspx">
311 NCover forums</a> as well as the <a href="http://ncover.org/SITE/blogs/default.aspx">
312 blog</a> by the author Peter Waldschmidt. If you cannot find a similar issue
313 mentioned feel free to post your query and perhaps someone can help.</P>
314 <P class="question">
315 12. How do I "xcopy deploy" NCover like my other build tools?</P>
316 <P class="answer">
317 Many developers prefer to have their build tools such as NUnit, NAnt etc stored
318 in source control in a Tools folder along with the source code. This ensures that
319 a new developer can obtain and build the application without having to install additional
320 tools on their own machines.</P>
321 <p>
322 NCover can also be deployed in this fashion. However the one gotcha with NCover
323 versus other tools is that the profiler within CoverLib.dll must be COM registered
324 on the local machine before you execute it. Prior to NCover 1.5.6 this was usually
325 achieved as part of your build script, which would call regsvr32 with the path to
326 the CoverLib.dll in your Tools folder. Alternatively the <ncover> NAnt and
327 MSBuild tasks described below will do this for you. As of NCover 1.5.6 you can also
328 use the //reg option in the command line arguments which will temporarily register
329 the profiler. Note that the //reg option will not work for IIS or Windows Service
330 profiling unless you are running NCover under the same Windows login account as
331 the IIS worker process, or your Windows Service.</p>
332 <P class="question">
333 13. How do I see my source code highlighted with the coverage results?</P>
334 <P class="answer">
335 <a href="http://ncoverexplorer.org/">NCoverExplorer</a> is a gui and console-based
336 .NET application developed by <a href="http://www.kiwidude.com/blog/">Grant Drake</a>. NCoverExplorer
337 parses the coverage.xml files output from NCover and displays the results integrated
338 with your source code. It also includes a number of additional features to merge,
339 filter, sort and generate html reports. The console version is
340 designed to be used as part of an automated build process. The support forums for
341 NCoverExplorer are located with the NCover ones at <a href="http://ncover.org">http://ncover.org/</a>.<strong> </strong></P>
342 <P class="question">
343 14. How do I run NCover from within the Visual Studio.Net IDE?</P>
344 <P class="answer">
345 The <a href="http://testdriven.net/">TestDriven.Net</a> add-in by <a href="http://weblogs.asp.net/nunitaddin/">
346 Jamie Cansdale</a> offers a right-click capability within the IDE to execute
347 your unit tests with code coverage. The results of the NCover code coverage are
348 displayed with the bundled NCoverExplorer gui for analysis and reporting.</P>
349 <P class="question">
350 15. How do I run NCover from a NAnt or MSBuild task?</P>
351 <P class="answer">
352 You can use an <exec> task with <a href="http://nant.sourceforge.net/">NAnt</a>
353 or an <Exec> task with MSBuild. Alternatively you may want to use the custom
354 <ncover> task for NAnt or <NCover> task for MSBuild developed by Grant
355 Drake for a more developer friendly syntax. The source code, compiled assemblies
356 and documentation are located in the NCoverExplorer.Extras.zip available from <a
357 href="http://ncoverexplorer.org/">http://ncoverexplorer.org/</a>.</P>
358 <P class="question">
359 16. How do I include NCover output in my CruiseControl.Net build reports?</P>
360 <P class="answer">
361 <a href="http://ccnet.thoughtworks.com/">CruiseControl.Net</a> is a continuous integration
362 build server which offers web-based reporting of the outputs of a build such as
363 unit test results and code coverage reporting. The default CruiseControl.Net installation
364 includes a basic stylesheet which works in combination with the standard coverage.xml
365 formatted output. So all you need to do is include the execution of NCover as part
366 of your build, then add a CruiseControl.Net merge file publisher task to integrate
367 the coverage.xml results into the build output.</P>
368 <p class="answer">
369 An improvement on the above to display more attractive and powerful reports as well
370 as minimize the build log size is to use NCoverExplorer. The NCoverExplorer.Console.exe
371 is designed to produce a more concise xml report summary that is combined with an
372 alternate xsl stylesheet for CruiseControl.Net. You can find more information and
373 screenshots in this <a href="http://www.kiwidude.com/blog/2006/04/ncoverexplorer-v133.html">
374 blog entry</a> - all the necessary tasks, examples and documentation are located
375 within NCoverExplorer.Extras.zip available from <a href="http://ncoverexplorer.org/">
376 http://ncoverexplorer.org/</a>. </p>
377 <P class="question">
378 17. How do I merge multiple NCover coverage.xml results?</P>
379 <P class="answer">
380 You can can use NCoverExplorer to merge the results of multiple coverage runs. For
381 more information refer to this <a href="http://www.kiwidude.com/blog/2006/10/ncoverexplorer-merging-ncover-reports.html">
382 blog entry</a>.</P>
383 <P class="question">
384 18. Troubleshooting: Why is my coverage.xml file empty?</P>
385 <ul>
386 <li>If using the command-line, did you COM register CoverLib.dll (or use the //reg option
387 from NCover 1.5.6)?</li>
388 <li>Did you generate build symbol files (.pdbs) for the profiled application?</li>
389 <li>If using the //a option, did you correctly list just the assembly names without
390 paths or .dll suffixes?</li>
391 </ul>
392 <P class="question">
393 19. Troubleshooting: I have coverage.xml output but my XYZ assembly is not included in it?</P>
394 <ul>
395 <li>NCover will only profile loaded assemblies - did your code execution path while
396 under coverage force that assembly to be loaded (e.g. by loading a type or calling
397 a method in that assembly)? </li>
398 <li>Did you generate build symbol files (.pdb files) for the missing assembly? </li>
399 <li>If using the //a option, did you correctly list the assembly names including the
400 one that is missing?</li>
401 <li>Can you see information about the assembly being loaded within the coverage.log?
402 Is the correct assembly being loaded (check the path) - if you have a version in
403 the GAC it may possibly prevent the .pdb file from being loaded.</li><li>If using the NCoverExplorer gui, have you got a coverage exclusion defined which
404 is hiding it from the display?</li>
405 </ul>
406 <P class="question">
407 20. Troubleshooting: After running NCover my coverage.log says "Failed to load symbols for module XYZ"?</P>
408 <ul>
409 <li>This message means that no .pdb build symbol file was found for that assembly so
410 it cannot be profiled for code coverage. If that assembly is part of the .NET framework
411 for instance like System.Data.dll, then this is an expected message and should not
412 cause concern. </li><li>If however the assembly belongs to your application, did you generate the
413 build symbol files (.pdb files) for it? </li>
414 </ul>
415 <P class="question">
416 21. Troubleshooting: I get a "Profiled process terminated. Profiler connection not
417 established" message?</P>
418 <ul>
419 <li>If using the command-line, did you COM register CoverLib.dll (or use the //reg option
420 from NCover 1.5.6)?</li><li>Are you running Windows XP 64-bit? You may want to take a look at
421 <a href="http://ncover.org/SITE/forums/thread/43.aspx">this thread</a></li></ul>
422 <P class="question">
423 22. Troubleshooting: My coverage exclusions are not working?</P>
424 <ul>
425 <li>Have you put the full namespace type name to the exclusion including the Attribute suffix in the //ea argument? See the "How
426 do I use coverage exclusions?" question above.</li></ul>
427
428 </body>
429</html>