OWASP O2 Platform Blog

Fortify FVDL files – Creating and consuming the schema and CSharp file

An o2 user send me a nice 400Mb Fortify file to see if O2 could do something with it (since Fortify’s GUI was not able to load it).

The next couple posts will document how to go from an XML file to creating an API to consume and to visualize it.

For this demo I will use the Fortify FVDL files that where published by SATE 2008 results (part of NIST SAMATE project) which Fortify participated. You can download the entire SATE 2008 data from their website (which includes the findings files from all participats and SATE’s result), or you can download just the Fortify FVDL (and xsd) from O2’s S3 repository: Fortify-Sate-2008.zip (the scripts below will use this files, but if you have access to *.fvdl files, you can use them)

To start, open an ‘O2 Quick Development Environment GUI’

Then download the demos files and unzip them to a local temp folder:

var demoFileUrl = "http://s3.amazonaws.com/Demo_Files/Fortify-Sate-2008.zip";
var localFile = demoFileUrl.uri().download();

Next step is to unzip the file:

var localFile = "".tempDir().pathCombine("Fortify-Sate-2008.zip");
if (localFile.fileExists().isFalse())
{
    var demoFileUrl = "http://s3.amazonaws.com/Demo_Files/Fortify-Sate-2008.zip";
    demoFileUrl.uri().download();
}   
return localFile.unzip_File();

Then add a check to only unzip if the target folder doesn’t exist and package it in a Lambda method which can be easily consumed by the main script:

Func<string> getFolderWithFvdlDemoFiles =
    ()=>{
            var localFile = "".tempDir().pathCombine("Fortify-Sate-2008.zip");
            if (localFile.fileExists().isFalse())
            {
                var demoFileUrl = "http://s3.amazonaws.com/Demo_Files/Fortify-Sate-2008.zip";
                demoFileUrl.uri().download();
            }    
            var targetFolder = @"..\_Fortify-Sate-2008".tempDir(false).fullPath(); // by default this willl resolve to C:\O2\_tempDir\_Fortify-Sate-2008
            if (targetFolder.dirExists().isFalse())
                localFile.unzip_File(targetFolder);            
            return targetFolder;
        };

var folderWithFvdlFiles = getFolderWithFvdlDemoFiles();       
return folderWithFvdlFiles.files(true,"*.fvdl");

There a number of *.fvdl files available, and since the first thing we need is to create an *.xsd for them, let’s pick the smaller one (in this case naim.fvdl)

var folderWithFvdlFiles = getFolderWithFvdlDemoFiles();        
var naimFvdl = folderWithFvdlFiles.files(true,"naim.fvdl").first();
return naimFvdl;

Usually at this stage we can use the O2 Fluent XML Apis to do this. Basically, in most cases this works:

var folderWithFvdlFiles = getFolderWithFvdlDemoFiles();        
var naimFvdl = folderWithFvdlFiles.files(true,"naim.fvdl").first();


return naimFvdl.xmlCreateXSD(); // creates XSD from Xml file

Unfortunately, for this XML file, we get the error “The table (Node) cannot be the child table to itself in nested relations.” (which is quite a common error when creating XSDs from XML files)

The 2nd usual attempt is to try to use the XSD.exe tool that comes with Visual Studio, But that just returns the same error:

The 3rd attempt is to use VisualStudio 2010 for the conversion. Before we open the file in visual studio, we will need to create a new copy (or rename it) with an .xml extension (ie C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.xml)

 Then from the XML menu, select the ‘Create Schema’ menu Item

Which this time around should work 🙂 :

Save this file locally

Now go back to the O2 development Gui (or open up a new one), and lets create the CSharp file. Start by making sure you have a correct reference to the file:

var xsdFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.xsd";
return xsdFile.fileExists();

call the xsdCreateCSharpFile method to create a CSharp file:

var xsdFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.xsd";
return xsdFile.xsdCreateCSharpFile(); 

This will create a file called ‘C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.cs’, which you can take look by opening it in a source code editor control

var topPanel = panel.clear().add_Panel();
var xsdFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.xsd";
var csharpFile = xsdFile.xsdCreateCSharpFile(); 
var codeEditor = topPanel.add_SourceCodeEditor(); // add source code editor to topPanel
codeEditor.invoke("setMaxLoadSize","1000");       // invoke the private method that sets the max size of the file to load using the SharpDevelop control (if bigger the file is opended in a listview)
codeEditor.open(csharpFile);                        // open file

if try to compile this csharp file, you will get a number of “The type of namespace ‘xyz’ could not be found”  errors

This is caused because the CSharp file was created using the Link2Xml Apis which are included in the O2_Misc_Microsoft_MPL_Libs.dll.

The solution is to add this as a reference at the top of the file, which will make the compilation work

Note that this extra reference could had also be introduced programatically like this:

var xsdFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.xsd";
var csharpFile = xsdFile.xsdCreateCSharpFile(); 

csharpFile.fileContents()
      .insertBefore("//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll".line())
      .saveAs(csharpFile);

The final step, now that we have the CSharp file (‘C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.cs’), is to use it as a reference and load up the original Xml file using it.

To make it clear, lets do this on a clean O2 Development GUI Environment

If you look at the CSHarp file you should notice that there is namespace

namespace xmlns.www.fortifysoftware.com.schema.fvdl {

…which contains a FVDL class

public partial class FVDL : XTypedElement, IXMetaData {

…which contains a Load static method

        public static FVDL Load(string xmlFile) {
            return XTypedServices.Load<FVDL>(xmlFile);
        }

This is the method that we will need to call , so that we can get a strongly typed version of our XML file

var xmlFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
return fvdl;

//using xmlns.www.fortifysoftware.com.schema.fvdl
//O2File:C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

 … which finally gives us the object that we want:

…and provides strongly-typed access to the fvdl data, including code-complete support

  ->

As a first exampe,  here is the list of vulnerability objects in the loaded xml file:

Note: one way to see in more details the type of objects that are availble is to call the {object}.details() method:

var xmlFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
var vulnerabilities = fvdl.Vulnerabilities.Vulnerability; 
vulnerabilities.details();

..which opens a popup window that can be navigated by properties or fields (note that the data is only loaded on selection)

Going back into Vulnerability filtering …

… where it gets really powerful is when we use Linq queries to filter the data:

var xmlFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
var vulnerabilities = fvdl.Vulnerabilities.Vulnerability; 

return (from vulnerability in vulnerabilities
        select vulnerability.ClassInfo.Kingdom).Distinct(); 

In this case, here is a distinct list of the vunerability’s Kingdoms

…  or a quick consolidated view of  the vulnerability data:

var topPanel = panel.clear().add_Panel();
var xmlFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
var vulnerabilities = fvdl.Vulnerabilities.Vulnerability; </pre>
&nbsp;

var results =  (from vulnerability in vulnerabilities
                  select new  {
                                kingdom = vulnerability.ClassInfo.Kingdom,
                                analyzer = vulnerability.ClassInfo.AnalyzerName,
                                classId = vulnerability.ClassInfo.ClassID,
                                defaultSeverity = vulnerability.ClassInfo.DefaultSeverity,
                                instanceId = vulnerability.InstanceInfo.InstanceID,
                                instanceSeverity = vulnerability.InstanceInfo.InstanceSeverity,
                                confidence = vulnerability.InstanceInfo.Confidence,
                                function = vulnerability.AnalysisInfo.Unified.Context.Function.name,
                                file = vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.path,
                                line = vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.line,
                                col = vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.colStart
                               
                            });
return results;

 

This  ‘Linq result’ object, is a local C# anonymous class, and is better seen and analyed on a TableList:

var topPanel = panel.clear().add_Panel();
var xmlFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
var vulnerabilities = fvdl.Vulnerabilities.Vulnerability; </pre>
&nbsp;

var results =  (from vulnerability in vulnerabilities
                  select new  {
                                kingdom = vulnerability.ClassInfo.Kingdom,
                                analyzer = vulnerability.ClassInfo.AnalyzerName,
                                classId = vulnerability.ClassInfo.ClassID,
                                defaultSeverity = vulnerability.ClassInfo.DefaultSeverity,
                                instanceId = vulnerability.InstanceInfo.InstanceID,
                                instanceSeverity = vulnerability.InstanceInfo.InstanceSeverity,
                                confidence = vulnerability.InstanceInfo.Confidence,
                                function = vulnerability.AnalysisInfo.Unified.Context.Function.name,
                                file = vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.path,
                                line = vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.line,
                                col = vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.colStart
                               
                            }).toList();

topPanel.add_TableList("Showing {0} Vulnerabilties".format(results.size()))
        .show(results);                           
return "done";

//using xmlns.www.fortifysoftware.com.schema.fvdl
//O2File:C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

That wraps up this example, the next related entries will continue from here 🙂

July 16, 2011 - Posted by | Fortify, Interoperability

3 Comments »

  1. […] from the (Fortify FVDL related) Creating and consuming the schema and CSharp file and Simple TableList Viewer Tool this post shows the next evolutionary step which is the creation […]

    Pingback by Fortify FVDL files – Creating an API and consumining it « OWASP O2 Platform Blog | July 17, 2011 | Reply

  2. […] the previous Fortify FVDL posts (here,  here, here and here), here is a first working tool that is able to load up *.fvdl files, parse […]

    Pingback by Fortify FVDL Files – First working Parser and Viewer for *.fvdl files « OWASP O2 Platform Blog | July 18, 2011 | Reply

  3. […] in this case, the script (sent to me by an O2 user who was following the code samples in the  Fortify FVDL files – Creating and consuming the schema and CSharp file blog post) is actually jumping a step (btw, see  here for more posts about O2 support for […]

    Pingback by O2 Scripting – solving the missing references compilation problem (using FVDL as an example) « OWASP O2 Platform Blog | July 28, 2011 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: