OWASP O2 Platform Blog

Creating a Pink version of IBM Rational AppScan Standard

Once you have an O2 script inside AppScan Standard what can you do with it?

(see Injecting O2 into IBM Rational AppScan Standard for details on how it was done)

Well, what about creating a pink version of AppScan (for the female users out there)?

How was this done?

It is actually very easy. Since the O2 Script editor is running under the same process as AppScan Standard, and there is a reference passed into the script envionment with the main GUI (the variable is called MainForm), all we need to do a recursive search for all Windows Controls and change its background color:

foreach(var controls in MainForm.controls(true))
    controls.backColor(Color.Pink);
MainForm.set_Text("AppScan Standard - in Pink");
return MainForm;
//O2Ref:appscan.exe

Over the next weeks I will post more examples on what can be done with these capabilities (namely the ability to add the artifacts that O2 is able to create/consume from other tools or from the target application’s Source Code)

Meanwhile, to help with your first scripts inside AppScan standard, here are a number of code samples that I captured during the development of these scripts:

Get AppScan.Exe assembly object (before AppScan starts)

return "AppScan.exe".assembly()

… get its types

return "AppScan.exe".assembly().types();

… get the MainForm class:

return "AppScan.exe".assembly()
                    .type("MainForm");

… get MainForm class methods:

return "AppScan.exe".assembly()
                    .type("MainForm")
                    .methods();

… get the Main method

return "AppScan.exe".assembly()
                    .type("MainForm")
                    .method("Main");

… invoke the Main method in an STA Thread

O2Thread.staThread(
    ()=>{
            "AppScan.exe".assembly()
                    .type("MainForm")
                    .method("Main").invoke(new object[] {new string[] {} }) ; 
        });

Getting a refrence to the main window and changing its title (after AppScan starts and inside the AppScan process)

var appScan =  "".lastFormLoaded();
appScan.set_Text("O2 version of IBM AppScan - in Pink");

…getting the top level controls:

return appScan.controls();

…getting the list of all GUI controls

return appScan.controls(true);

… getting all treeviews

return appScan.controls<TreeView>(true);

Adding a test node to all treeviews

var treeViews = appScan.controls<TreeView>(true);
foreach(var treeview in treeViews)
    treeview.add_Node("Hello from O2");
return treeViews;

Grabbing a reference to the top Level menu and creating an O2 Menu

var menuStrip =  appScan.controls<MenuStrip>();
var o2Menu = menuStrip.add_MenuItem("O2");

Adding a menu item that opens up an O2 Log Viewer

var appScan =  "".applicationWinForms()[2];
var menuStrip =  appScan.controls<MenuStrip>(); 
var o2Menu = menuStrip.add_MenuItem("O2"); 
o2Menu.add_MenuItem("O2 Log Viewer", ()=>  "O2 Log Viewer".popupWindow(450,200).add_LogViewer() );


Add menu item that opens up a script editor

var appScan =  "".applicationWinForms()[3];
Action<Panel> showO2ScriptEditor =
    (targetPanel)=> {
                        var scriptEditor = targetPanel.add_Script(false) ;
                        scriptEditor.InvocationParameters.add("MainForm", appScan);
                        scriptEditor.Code = "return MainForm;".line() +
                                            "\\O2Ref:appscan.exe";
                    };

var menuStrip =  appScan.controls<MenuStrip>(); 
var o2Menu = menuStrip.add_MenuItem("O2"); 
o2Menu.add_MenuItem("O2 Log Viewer", ()=>  "O2 Log Viewer".popupWindow(450,200).add_LogViewer() );
o2Menu.add_MenuItem("O2 Script Editor", ()=>  showO2ScriptEditor("O2 Script Editor".popupWindow(650,300)) );

August 6, 2011 Posted by | AppScan, Tools | Leave a comment

Injecting O2 into IBM Rational AppScan Standard

If you use AppScan Standard (the BlackBox tool) and want to use O2 to script its GUI or scans, here is a new O2 script that will start AppScan Standard under the same process as O2.

Once the main AppScan gui is loaded this script will add a new O2 Menu and inject an O2 Script editor.

You can find the script that automates the process in your local folder C:\O2\O2Scripts_Database\_Scripts\3rdParty_Tools\IBM\AppScan_Standard:

The script you want to execute is the Util – Launch AppScan Standard (O2 version).h2  which will

  • Copy all required files (O2 dlls and AppScan_Standard_1st_Script.cs Script) to the AppScan Folder.
  • Compile the AppScan_Standard_O2_Launcher.cs script into AppScan_Standard_O2_Launcher.exe and copy it to AppScan folder
  • In the AppScan folder: execute and compile AppScan_Standard_O2_Launcher.exe
  • The AppScan_Standard_O2_Launcher.exe will compile the AppScan_Standard_1st_Script.cs and execute it 
  • the AppScan_Standard_1st_Script.cs script will:
    • open an O2 Log Viewer (so that you get a feel for what is happening),
    • launch AppScan (by using reflection to open the MainForm Form control),
    • wait for the main AppScan gui to load, and when it does:
    • compile and execute the In AppScan – Create O2 Gui.h2 script
  • The In AppScan – Create O2 Gui.h2  script is the one that adds a new menu item and injects a O2 Script editor into the main AppScanGui.

The Util – Launch AppScan Standard (O2 version).h2  has an editor for all these scripts, but by default this is disabled (see NUnit example for details). The default behaviour of Util – Launch AppScan Standard (O2 version).h2 (when double-clicked from windows explorer) is to execute the above steps in sequence. And if all goes according to plan, you will see the following ‘AppScan Standard – O2 Version’ GUI:

… note the extra O2 Menu and the O2 Scripting Environment….

 

For reference, here is what it looks like while AppScan Standard is loading up (the cmd.exe window was created by AppScan_Standard_O2_Launcher.exe and the LogViewer shows the wait for the main AppScan GUI to load up)

 

These series of scripts are heavily based on the ones previously used and documented in the Injecting O2 into another .NET Process (in this case NUnit.exe) script (see that blog post for more technical details about how this works).

August 6, 2011 Posted by | .NET, AppScan, Interoperability | 1 Comment

Creating the the “Util – View Fvdl Traces.h2” script (lots of data analysis code samples)

If you are trying to write a script to filter Fortify’s FVDL files, here are a number of code snippets that should give you a good idea of the type of scripting and analysis that you can do with O2.

This workflow represents the creation of a GUI to visualize an FVDL file. You can see the end result in the Util – View Fvdl Traces.h2 script which can be found in the following local O2 scripts folder

This is what the script looks like when executed for the first time

And this is what it looks like after an *.fvdl file is dropped in the top left treeview:

… and (after load)

 

Here is the workflow and code snippets:

Start with:

var topPanel = panel.clear().add_Panel();
var fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fortifyScan = fvdlFile.fortifyScan();
topPanel.add_PropertyGrid()
.helpVisible(false)
.show(fortifyScan);
return fortifyScan.typeName();
//O2File:API_Fortify_1_6.cs

add treeview

var fortifyScan = fvdlFile.fortifyScan();
var treeView = topPanel.add_TreeView_with_PropertyGrid(false);

add Vulnerabilities in treeview and show details in propertygrid

var fortifyScan = fvdlFile.fortifyScan();
var treeView = topPanel.add_TreeView_with_PropertyGrid(false);
treeView.add_Nodes(fortifyScan.Vulnerabilities);
treeView.selectFirst();

Get list of unique ‘Confidence’ values

return (from vulnerability in fortifyScan.Vulnerabilities
       select vulnerability.Confidence).Distinct();

Get list of unique ‘InstanceSeverity’ values

return (from vulnerability in fortifyScan.Vulnerabilities 
       select vulnerability.InstanceSeverity).Distinct();

Create dictionary of vulnerabilties mapped by ‘InstanceSeverity’

var fortifyScan = fvdlFile.fortifyScan();
var mappedBySeverity = new Dictionary<decimal, List<Fortify_Vulnerability>>();
foreach(var vulnerability in fortifyScan.Vulnerabilities)
    mappedBySeverity.add(vulnerability.InstanceSeverity, vulnerability);
return mappedBySeverity;

Add dictionary to treeView (loads all data)

foreach(var item in mappedBySeverity)
    treeView.add_Node("{0}        ({1} vulnerabilities)".format(item.Key.str(),item.Value.size()))
            .add_Nodes(item.Value);
treeView.selectFirst();

Add dictionary to treeView (with support for lazy loading of vulnerabilities (beter for bigger files))

foreach(var item in mappedBySeverity)
    treeView.add_Node("{0}        ({1} vulnerabilities)".format(item.Key.str(),item.Value.size()),
                     item.Value,
                     true);
                    
treeView.beforeExpand<List<    Fortify_Vulnerability>>(
    (treeNode,vulnerabilities)=>{
                                    treeNode.add_Nodes(vulnerabilities);
                                });
                               
treeView.selectFirst()
        .selected()
        .expand();
return mappedBySeverity;

populate treeview with two types of dictionaries (first ‘by Severity’ then ‘by Type’)

var mappedBySeverity = new Dictionary<decimal, List<Fortify_Vulnerability>>();
foreach(var vulnerability in fortifyScan.Vulnerabilities)
    mappedBySeverity.add(vulnerability.InstanceSeverity, vulnerability);
 
 
foreach(var bySeverity in mappedBySeverity)
{
    var mappedByType = new Dictionary<string, List<Fortify_Vulnerability>>();
    foreach(var vulnerability in bySeverity.Value)
        mappedByType.add(vulnerability.Type, vulnerability);           
    var severityNode = treeView.add_Node("{0}      ({1}x)".format(bySeverity.Key.str(),bySeverity.Value.size()));
    foreach(var byType in mappedByType)               
         severityNode.add_Node("{0}  ({1}x)".format(byType.Key, byType.Value.size()), byType.Value, true);
}
treeView.beforeExpand<List<    Fortify_Vulnerability>>(
    (treeNode,vulnerabilities)=>{
                                    treeNode.add_Nodes(vulnerabilities);
                                });
                               
treeView.selectFirst()
        .selected()
        .expand();

Move dictionary and treeView creation to Lambda method (same funcionality as previous example)

Action<Fortify_Scan, TreeView> mapScanToTreeView_BySeverityAndType =
    (fortifyScan, treeView)
        =>{
                treeView.clear();
                var mappedBySeverity = new Dictionary<decimal, List<Fortify_Vulnerability>>();
                foreach(var vulnerability in fortifyScan.Vulnerabilities)
                    mappedBySeverity.add(vulnerability.InstanceSeverity, vulnerability);
                              
                foreach(var bySeverity in mappedBySeverity)
                {
                    var mappedByType = new Dictionary<string, List<Fortify_Vulnerability>>();
                    foreach(var vulnerability in bySeverity.Value)
                        mappedByType.add(vulnerability.Type, vulnerability);           
                    var severityNode = treeView.add_Node("{0}      ({1}x)".format(bySeverity.Key.str(),bySeverity.Value.size()));
                    foreach(var byType in mappedByType)                                       
                        severityNode.add_Node("{0}  ({1}x)".format(byType.Key, byType.Value.size()), byType.Value, true);
                }
               
                treeView.beforeExpand<List<    Fortify_Vulnerability>>(
                    (treeNode,vulnerabilities)=>{
                                                    treeNode.add_Nodes(vulnerabilities);
                                                });
        };
var fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var _treeView = topPanel.add_TreeView_with_PropertyGrid(false).sort();
var _fortifyScan = fvdlFile.fortifyScan();

mapScanToTreeView_BySeverityAndType(_fortifyScan, _treeView);

_treeView.focus()
         .selectFirst()
         .selected()
         .expand();

Add a treeView to right and a source code viewer bellow it

var fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var _treeView = topPanel.add_GroupBox("Vulnerabilities results").add_TreeView_with_PropertyGrid(true).sort();
var _fortifyScan = fvdlFile.fortifyScan();
var tracesTreeView = topPanel.insert_Right("Traces").add_TreeView();
var codeViewer = tracesTreeView.parent().insert_Below("Source Code View").add_SourceCodeViewer();

mapScanToTreeView_BySeverityAndType(_fortifyScan, _treeView);

Show traces on Vulnerability select , show traces properties on trace select (using the dspace.fvdl file since it is a java scan and it has more traces) 

//var fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\dspace.fvdl";
var _treeView = topPanel.add_GroupBox("Vulnerabilities results").add_TreeView_with_PropertyGrid(true).sort();
_treeView.add_Node(".... loading file: {0}".format(fvdlFile));
var _fortifyScan = fvdlFile.fortifyScan();
var tracesTreeView = topPanel.insert_Right("Traces").add_TreeView();
var tracesProperties = tracesTreeView.insert_Right().add_PropertyGrid();
var codeViewer = tracesTreeView.parent().insert_Below("Source Code View").add_SourceCodeViewer();

_treeView.afterSelect<Fortify_Vulnerability>(
    (vulnerability)=>{                       
                        tracesTreeView.clear();
                        tracesTreeView.add_Nodes(vulnerability.Traces);
                     });
tracesTreeView.afterSelect<Fortify_TraceEntry>(
    (traceEntry)=>{
                    tracesProperties.show(traceEntry); 
                  });
                
mapScanToTreeView_BySeverityAndType(_fortifyScan, _treeView);
_treeView.focus()
         .selectFirst()
         .selected()
         .expand().nodes()[0].selected()
         .expand().nodes()[0].selected();

tracesTreeView.selectFirst();

Move data loading into a separate thread

O2Thread.mtaThread(
    ()=>{
            var _fortifyScan = fvdlFile.fortifyScan();
            mapScanToTreeView_BySeverityAndType(_fortifyScan, _treeView);
           
            _treeView.focus()
                     .selectFirst()
                     .selected()
                     .expand().nodes()[0].selected()
                     .expand().nodes()[0].selected();
           
            tracesTreeView.selectFirst();       
        });

and move it to a Lambda method (with callback for when data is loaded)

Action<string, TreeView, Action> loadFvdlDataIntoTreeView =
    (fvdlFile, vulnerabilitiesTreeView, loadCompleteCallback)
        =>{
            vulnerabilitiesTreeView.add_Node(".... loading file: {0}".format(fvdlFile));           
            vulnerabilitiesTreeView.pink();
            O2Thread.mtaThread(
                ()=>{                       
                        var _fortifyScan = fvdlFile.fortifyScan();
                        mapScanToTreeView_BySeverityAndType(_fortifyScan, vulnerabilitiesTreeView);
                        loadCompleteCallback();
                        vulnerabilitiesTreeView.white();
                    });
         };
var _fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\dspace.fvdl";
//var fvdlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
loadFvdlDataIntoTreeView(_fvdlFile,_treeView,
        ()=>{
               
                _treeView.focus()
                         .selectFirst()
                         .selected()
                         .expand().nodes()[0].selected()
                         .expand().nodes()[0].selected();               
                tracesTreeView.selectFirst();            
            });

add support for Code Snippets

var codeSnippets = new Dictionary<string, Fortify_Snippet>();
Action<Fortify_Scan> loadCodeSnippets =
    (_fortifyScan)=>{
                        codeSnippets.Clear();
                        foreach(var snippet in _fortifyScan.Snippets)
                            codeSnippets.Add(snippet.Id, snippet);                   
                        show.info(codeSnippets);                   
                    };
...
var _fortifyScan = fvdlFile.fortifyScan();
loadCodeSnippets(_fortifyScan);

Show code snippet when trace is selected

tracesTreeView.afterSelect<Fortify_TraceEntry>(
    (traceEntry)=>{
                    tracesProperties.show(traceEntry); 
                    if (codeSnippets.hasKey(traceEntry.SourceLocation_Snippet))
                        codeViewer.set_Text(codeSnippets[traceEntry.SourceLocation_Snippet].Text);
                    else
                        if (codeSnippets.hasKey(traceEntry.SecundaryLocation_Snippet))
                        codeViewer.set_Text(codeSnippets[traceEntry.SecundaryLocation_Snippet].Text);
                   
                  });

Only show traces that have valid code snippets

_treeView.afterSelect<Fortify_Vulnerability>(
    (vulnerability)=>{                       
                        tracesTreeView.clear();
                        var tracesWithSnippets = (from traceEntry in vulnerability.Traces
                                                where traceEntry.SourceLocation_Snippet.valid() || traceEntry.SecundaryLocation_Snippet.valid()
                                                select traceEntry).Distinct;
                        tracesTreeView.add_Nodes(tracesWithSnippets);
                     });

Add DragAndDrop support

Action<string> loadFvdlFile =
    (fvdlFile)=>{
                    loadFvdlDataIntoTreeView(fvdlFile,_treeView,
                            ()=>{
                                   
                                    _treeView.focus();
                                /*             .selectFirst()
                                             .selected()
                                             .expand().nodes()[0].selected()
                                             .expand().nodes()[0].selected();               
                                    tracesTreeView.selectFirst();             */
                                });
                };
_treeView.onDrop(
    (file)=>{
                loadFvdlFile(file);
            });

and open the GUI in a new window

var topPanel = "Util - View Fvdl Traces".popupWindow(1000,400);

FOR REFERENCE: Here is the complete script:

var topPanel = "Util = View Fvdl Traces".popupWindow(1000,400);
//var topPanel = panel.clear().add_Panel();</pre>
&nbsp;

Action<Fortify_Scan, TreeView> mapScanToTreeView_BySeverityAndType =
    (fortifyScan, treeView)
        =>{
                treeView.clear();
                var mappedBySeverity = new Dictionary<decimal, List<Fortify_Vulnerability>>();
                foreach(var vulnerability in fortifyScan.Vulnerabilities)
                    mappedBySeverity.add(vulnerability.InstanceSeverity, vulnerability);
               
               
                foreach(var bySeverity in mappedBySeverity)
                {
                    var mappedByType = new Dictionary<string, List<Fortify_Vulnerability>>();
                    foreach(var vulnerability in bySeverity.Value)
                        mappedByType.add(vulnerability.Type, vulnerability);           
                    var severityNode = treeView.add_Node("{0}      ({1}x)".format(bySeverity.Key.str(),bySeverity.Value.size()));
                    foreach(var byType in mappedByType)                                       
                        severityNode.add_Node("{0}  ({1}x)".format(byType.Key, byType.Value.size()), byType.Value, true);
                }
               
                treeView.beforeExpand<List<    Fortify_Vulnerability>>(
                    (treeNode,vulnerabilities)=>{
                                                    treeNode.add_Nodes(vulnerabilities);
                                                });
        };                               

var _treeView = topPanel.add_GroupBox("Vulnerabilities results").add_TreeView_with_PropertyGrid(true).sort();
var tracesTreeView = topPanel.insert_Right("Traces").add_TreeView();
var codeViewer = tracesTreeView.parent().insert_Below("Source Code View").add_SourceCodeViewer();
var tracesProperties = tracesTreeView.insert_Right().add_PropertyGrid();

_treeView.afterSelect<Fortify_Vulnerability>(
    (vulnerability)=>{                       
                        tracesTreeView.clear();
                        var tracesWithSnippets = (from traceEntry in vulnerability.Traces
                                                where traceEntry.SourceLocation_Snippet.valid() || traceEntry.SecundaryLocation_Snippet.valid()
                                                select traceEntry);
                        tracesTreeView.add_Nodes(tracesWithSnippets);
                     });

var codeSnippets = new Dictionary<string, Fortify_Snippet>();

Action<Fortify_Scan> loadCodeSnippets =
    (_fortifyScan)=>{
                        codeSnippets.Clear();
                        foreach(var snippet in _fortifyScan.Snippets)
                            codeSnippets.Add(snippet.Id, snippet);                                           
                    };
                   
tracesTreeView.afterSelect<Fortify_TraceEntry>(
    (traceEntry)=>{
                    tracesProperties.show(traceEntry); 
                    if (codeSnippets.hasKey(traceEntry.SourceLocation_Snippet))
                        codeViewer.set_Text(codeSnippets[traceEntry.SourceLocation_Snippet].Text);
                    else
                        if (codeSnippets.hasKey(traceEntry.SecundaryLocation_Snippet))
                        codeViewer.set_Text(codeSnippets[traceEntry.SecundaryLocation_Snippet].Text);
                   
                  });

Action<string, TreeView, Action> loadFvdlDataIntoTreeView =
    (fvdlFile, vulnerabilitiesTreeView, loadCompleteCallback)
        =>{
            vulnerabilitiesTreeView.add_Node(".... loading file: {0}".format(fvdlFile));           
            vulnerabilitiesTreeView.pink();
            O2Thread.mtaThread(
                ()=>{                       
                        var _fortifyScan = fvdlFile.fortifyScan();
                        loadCodeSnippets(_fortifyScan);
                        mapScanToTreeView_BySeverityAndType(_fortifyScan, vulnerabilitiesTreeView);
                        loadCompleteCallback();
                        vulnerabilitiesTreeView.white();
                    });
         };
Action<string> loadFvdlFile =
    (fvdlFile)=>{
                    loadFvdlDataIntoTreeView(fvdlFile,_treeView,
                            ()=>{
                                   
                                    _treeView.focus();
                                /*             .selectFirst()
                                             .selected()
                                             .expand().nodes()[0].selected()
                                             .expand().nodes()[0].selected();               
                                    tracesTreeView.selectFirst();             */
                                });
                };
_treeView.onDrop(
    (file)=>{
                loadFvdlFile(file);
            });

return "ok";

//O2File:API_Fortify_1_6.cs

July 29, 2011 Posted by | Fortify | Leave a comment

O2 Scripting – solving the missing references compilation problem (using FVDL as an example)

A common error when starting to write O2 scripts is the “The name ‘…’ does not exist in the current context” compilation problem. This tends to happen because the C# comments that provide the dll and namespace references were not used (since it can easy to assume that they are not needed 🙂  )

Here is how to replicate the problem and how to fix it.

Start with an new instance of an O2 Quick Development GUI

Right-click on the source code editor and chose the ‘show log view option)

Then (for example) paste the code below:

var topPanel = panel.clear().add_Panel();
var xmlFile = @"C:\O2\Fortify\test.fvdl";
var xsdFile = @"C:\O2\Fortify\test.fvdl.xsd";
var csharpFile = xsdFile.xsdCreateCSharpFile();
csharpFile.fileContents().insertBefore("//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll".line()).saveAs(csharpFile);
var fvdl = FVDL.Load(xmlFile);

… this will trigger the code compilation, which will fail with the error:

[8:24:04 AM] ERROR: [CSharp_FastCompiler] Compilation Error: 32::14::CS0103::The name ‘FVDL’ does not exist in the current context::c:\Users\o2\AppData\Local\Temp\meqvg7zx.0.cs

The reference that is missing is the C# file that has the FVDL class.

In fact, in this case, the script (sent to me by an O2 user who was following the code samples in the  Fortify FVDL files – Creating and consuming the schema and CSharp file blog post) is actually jumping a step (btw, see  here for more posts about O2 support for Fortify’s FVDL files)

Let’s comment the last line,  and make the code compile

Before we execute it, let’s change the script a little bit to create the XSD file and then return the value of the csharpFile variable (which is the C# file created that represents the XSD file)

var xmlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var xsdFile = xmlFile.xmlCreateXSD();
var csharpFile = xsdFile.xsdCreateCSharpFile();
return csharpFile.fileContents().insertBefore("//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll".line()).saveAs(csharpFile);

In this case (due to the format of the Fortify XSD), the XML -> XSD conversion using Linq2Xml will not work (see the post  Fortify FVDL files – Creating and consuming the schema and CSharp file for more details and for a solution using Visual Studio 2010):

To show how to fix the  “The name ‘…’ does not exist in the current context” problem, let go back to the orignal script, without the bits that are not needed (and using a file from the SATE 2008 project):

var topPanel = panel.clear().add_Panel();
var xmlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);

To compile this we need to add two things:

1) a C#file or dll that has the FVDL class
2) an include reference to the namespace that contains the FVDL class:

In this case we are going to use the file that was created in the previous blog post example, which is now included in the local O2 Scripts folder:

To consume this file (and tell the O2 scripting engine to compile it with the current script), use this syntax:

//O2File:C:\O2\O2Scripts_Database\_Scripts\3rdParty_Tools\Fortify\Fortify.fvdl.1.6.xsd.cs

The compilation process now will show a lot more activity,  including an entry with ‘Compilated OK to: C:\O2\_tempDir\7-29-2011\tmp3A4C.tmp.dll‘  , which basically means that the files we provided as references compiled ok.

The reason we sill get the original error is because we are missing the namespace reference (altough you can already gain acccess to the imported code/dll Code Complete :

… and here is the FVDL class:

… which when  used as a reference using this syntax:

//using xmlns.www.fortifysoftware.com.schema.fvdl

… will result in this error:

 … which is also a very common error, and basically means that we also need to provide the C# compiler a reference to the O2_Misc_Microsoft_MPL_Libs.dll.

This can be done using this syntax:

//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

which will make the following code

var xmlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);</pre>
&nbsp;

//using xmlns.www.fortifysoftware.com.schema.fvdl
//O2File:C:\O2\O2Scripts_Database\_Scripts\3rdParty_Tools\Fortify\Fortify.fvdl.1.6.xsd.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

 

… to compile OK :

Note that since the Fortify.fvdl.1.6.xsd.cs file is included inside the O2 Scripts folder, we can use it just its filename:

var xmlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
 
//using xmlns.www.fortifysoftware.com.schema.fvdl
//O2File:Fortify.fvdl.1.6.xsd.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

To see the object created add a return statement:

…. (since we are inside the O2 Quick Development Gui and have access to a panel),  add a PropertyGrid to the provided Panel control and show the FVDL object there:

var topPanel = panel.clear().add_Panel();
var xmlFile = @"C:\O2\Demos\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
var fvdl = FVDL.Load(xmlFile);
topPanel.add_PropertyGrid().show(fvdl);
 
//using xmlns.www.fortifysoftware.com.schema.fvdl
//O2File:Fortify.fvdl.1.6.xsd.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll</pre>
&nbsp;

 

And if want to make this into a stand alone tool (that can be shared with others and invoked directly from the windows explorer), replace with first line with a call to open a popup window

//var topPanel = panel.clear().add_Panel();   
var topPanel = "View FVDL file".popupWindow(500,370);

If you save this script as a stand alone tool, go to the script editor, right click on it, and chose the current source code ->  save As menu item

Chose a location:

and execute this script by double clicking on it:

You might have noticed that I saved this file on my local development box in one of the O2 Scripts folders.

This means that to add it to the O2 scripts that exist locally to O2 users (and that O2 automatically syncs up when the main GUI starts), I just need to do this:

and it is done:

 … by the time you are reading this blog post you should have this script on your local O2 Scripts folder

July 28, 2011 Posted by | Fortify, O2 Internals | Leave a comment

Submit file to Veracode Trial: using Browser and Windows Automation (WatiN and White APIs)

This script automates the process off submitting a file to Veracode’s free trial (as of 20/Jul/2011). See also Consuming Veracode Findings File(s) using O2.

If first provides a settings GUI, where the user can enter the requried data (email and file to upload), then fires up a web brower and uses WatiN (Browser Automation) plus White (Windows UIAutomation) to populate the form fields and to submit the form.

The reason why I had to use UIAutomation (and White API) was because there didn’t seem to be a way to modify the HTML form field of the ‘file upload’ control (even jQuery didn’t seem to be able to modify that value programatially).

The solution was to use UIAutomation to:

  • click on the ‘Browse’ field,
  • then (on the popup file dialog window)
    • enter the file to upload, and
    • click ‘Close’

Here is the ‘settings window:

When the button is clicked the veracode trial page is opened and the ‘Html Form File Upload Button’ is pressed (via UIAutomation) :

When the ‘Choose File to Upload’ window appears, the ‘File Name Text Box’ is populated and the Open Button is pressed (both using UIAutomation (via White API)).

Finally, the email field is populated, the check box is ticked and the ‘Upload’ button is pressed (using Browser Automation (WatiN))

Here is the source code of this script:

var topPanel = O2Gui.open<Panel>("Util - Submit file to Veracode Trial",700,180);    
//var topPanel = panel.clear().add_Panel();
topPanel.insert_LogViewer();
var _email = "o2@o2platform.com";
var _fileToUpload = @"C:\O2\Demos\jPetStore - O2 Demo Pack\apache-tomcat-7.0.16\webapps\jpetstore.war";


Action<string,string> submitFileToVeracode =
	(email, fileToUpload)=>
		{
			var windowName= "Veracode File Upload - {0}".format(10.randomLetters());
			var ie = windowName.popupWindow(1000,500)
							   .add_IE().open("https://trial.veracode.com/freetrials/veracode-free-trial-signup.php");

			var processId = Processes.getCurrentProcessID();
			var apiGuiAutomation = new API_GuiAutomation(processId);
			var window = apiGuiAutomation.window(windowName);
			"got main window".info();
			var buttons = window.buttons();a
			"found {0} buttons".info(buttons.size());
			buttons[1].mouse().click();
			"clicked button".debug();
			var selectFileWindow = apiGuiAutomation.window("Choose File to Upload");
			selectFileWindow.textBoxes()[0].set_Text(fileToUpload);
			selectFileWindow.button("Open").click();
			ie.field("email",email);
			ie.checkBoxes()[0].check();
			ie.button("Upload").click();
			//buttons[2].mouse().click();
		};


topPanel.add_TextBox("Email",_email).top(0)
			.width(100)
			.onTextChange((text)=>_email=text)
		.append_Label("File to upload")
			.autoSize()
			.top(3)
		.append_TextBox(_fileToUpload)
			.onTextChange((text)=>_fileToUpload=text)
			.width(300)
			.align_Right(topPanel);
topPanel.add_Button(24,"Create Account and Upload File")
		.font_bold()
		.align_Right(topPanel)
		.onClick(()=> submitFileToVeracode(_email, _fileToUpload) );

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll
//O2File:API_GuiAutomation.cs
//O2Ref:White.Core.dll

July 20, 2011 Posted by | Interoperability, JPetStore, Veracode, WatiN, White_UIAutomation | Leave a comment

Fortify FVDL Files – Simple Viewer based on PropertyGrid

The .NET PropertyGrid Control can be a very powerful way to visualize objects.

For exampe, the script below creates a WinForm with just a PropertyGrid Control (and some drag and drop support to load up the files)

var topPanel = "Util - FVDL viewer (just PropertyGrid) - drop file to load".popupWindow(400, 400);
topPanel.insert_LogViewer();
var propertyGrid = topPanel.add_PropertyGrid().helpVisible(false);   
Action<string> loadAndShowFile =   
    (file)=> O2Thread.mtaThread(
        ()=>    propertyGrid.show(new API_Fortify().convertToFortifyScan(file)));

propertyGrid.onDrop(loadAndShowFile);           
//O2File:API_Fortify_1_6.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll 

which looks like this when opened

…and like this after dropping the 36Mb dspace.fvdl file:

Since most Fortify_* objects have been designed as lists, the PropertyGrid will show them in special pop-up windows:

CalledWithNoDefs

Contexts

Descriptions

Scanned Files

Sinks

Snippets

Sources

Vulnerabilities

Vulnerabilities -> Traces

July 18, 2011 Posted by | Fortify, Interoperability | Leave a comment

Fortify FVDL Files – Looking at the API_Fortify classes that parse the fvdl data

Here is the source code that create the environment/GUI/tool shown in the Fortify FVDL Files – First working Parser and Viewer for *.fvdl files post

Heavily based on  the first version of this API , the data classes have been refactored and expanded, now covering most of the data contained in the *.fvdl files:

public class Fortify_Scan
    {
        public FVDL _fvdl;
       
        public string BuildID { get; set; }
        public uint Loc { get; set; }
        public string SourceBasePath { get; set; }
        public uint ScanTime { get; set; }       
        public DateTime CreatedDateTime { get; set; }
        public string Errors { get; set; }
        public List<Fortify_ScannedFile> ScannedFiles { get; set; }   
        public List<Fortify_Context> Contexts { get; set; }
        public List<Fortify_Description> Descriptions { get; set; }
        public List<Fortify_CalledWithNoDef> CalledWithNoDefs { get; set; }       
        public List<Fortify_Sink> Sinks { get; set; }
        public List<Fortify_Source> Sources { get; set; }
        public List<Fortify_Snippet> Snippets { get; set; }
        public List<Fortify_Vulnerability> Vulnerabilities { get; set; }
       
        public Fortify_Scan()
        {
            ScannedFiles = new List<Fortify_ScannedFile>();
            Contexts = new List<Fortify_Context>();
            Descriptions = new List<Fortify_Description>();
            CalledWithNoDefs = new List<Fortify_CalledWithNoDef>();
            Sinks = new List<Fortify_Sink>();
            Sources = new List<Fortify_Source>();
            Snippets = new List<Fortify_Snippet>();
            Vulnerabilities = new List<Fortify_Vulnerability>();
        }
    }       
   
    public class Fortify_ScannedFile
    {
        public uint Loc { get; set; }
        public uint Size { get; set; }
        public ulong Timestamp { get; set; }
        public string Type { get; set; }
        public string Path { get; set; }
    }
    public class Fortify_Context
    {
        public string Id { get; set; }
        public Fortify_Function Function { get; set; }
    }   
    public class Fortify_Function
    {
        public string FunctionName { get; set; }
        public Fortify_CodeLocation CodeLocation { get; set; }   
       
        public Fortify_Function()
        {}
               
        public Fortify_Function(string functionName, object location)
        {
            FunctionName =  functionName;   
            CodeLocation = new Fortify_CodeLocation(location);
        }
       
        public override string ToString()
        {
            return "{{ name: {0} , path: {1} , line: {2}, lineEnd: {3}, colStart: {4}, colEnd: {5} }}"
                        .format(FunctionName, CodeLocation.Path, CodeLocation.Line, CodeLocation.LineEnd, CodeLocation.ColStart, CodeLocation.ColEnd);
        }
    }   
    public class Fortify_CodeLocation
    {
        public string Path { get; set; }
        public uint Line { get; set; }
        public uint LineEnd { get; set; }
        public uint ColStart { get; set; }
        public uint ColEnd { get; set; }
       
        public Fortify_CodeLocation()
        {}
       
        public Fortify_CodeLocation(object location)
        {                                   
            Path = (string)location.prop("path");                                                           
            Line = (uint)location.prop("line");
            LineEnd = (uint)location.prop("lineEnd");
            ColStart = UInt16.Parse(location.prop("colStart").str());
            ColEnd = UInt16.Parse(location.prop("colEnd").str());                                                       
        }
        public override string ToString()
        {
            if (ColStart == 0 && ColEnd==0)
                return "{{ path: {0} , line: {1}, lineEnd: {2}}}".format(Path, Line, LineEnd);
               
            return "{{ path: {0} , line: {1}, lineEnd: {2}, colStart: {3}, colEnd: {4} }}"
                        .format(Path, Line, LineEnd, ColStart, ColEnd);
        }
    }   
    public class Fortify_Description
    {
        public string Abstract { get; set; }
        public string ClassID { get; set; }
        public string ContentType { get; set; }
        public string Explanation { get; set; }
        public string Recommendations { get; set; }
        public List<string> Tips { get; set; }       
       
        public string Tips_as_String { get { return Tips.join(" , ");} } //viewer helper
       
        public Fortify_Description()
        {
            Tips = new List<string>();
        }
       
    }   
   
    public class Fortify_CalledWithNoDef
    {
        public string Name { get; set; }
        public string Namespace { get; set; }
        public string EnclosingClass { get; set; }
    }
   
    public class Fortify_Sink
    {
        public string RuleID { get; set; }
        public Fortify_Function Function_Call{ get; set;}               
    }
    public class Fortify_Source
    {
        public string RuleID { get; set; }
        public Fortify_Function Function_Call{ get; set;}
        public Fortify_Function Function_Entry{ get; set;}               
        public List<string> TaintFlags { get; set;}
       
        public string TaintFlags_as_String { get { return TaintFlags.join(" , ");} } //viewer helper
       
        public Fortify_Source()
        {
            TaintFlags = new List<string>();
        }
    }       
    public class Fortify_Snippet
    {
        public string Id { get; set; }
        public Fortify_CodeLocation CodeLocation { get; set; }       
        public string Text { get; set;}       
    }
   
    public class Fortify_ReplacementDefinitions
    {
        public Items Definitions { get; set; }
        //public List<Fortify_Function> LocationDefinitions { get; set; }  // add later if needed (Fortify_Function is not the best since the LocationDef has a 'key'  value
       
        public Fortify_ReplacementDefinitions()
        {
            Definitions = new Items();           
        }
        public override string ToString()
        {   
            return "{0} definitions".format(Definitions.size());
        }
    }
   
    public class Fortify_TraceEntry
    {
        public uint NodeRefId                                 { get; set; }    // when this one is set, the ones below are not set                       
        public string ActionType                             { get; set; }
        public string ActionValue                             { get; set; }
        public bool DetailsOnly                             { get; set; }
        public bool IsDefault                                 { get; set; }       
        public List<Fortify_TraceEntryFact> KnowledgeFacts    { get; set; }
        public string Reason_RuleId                            { get; set; }
        public string Reason_TraceRef                        { get; set; }       
        public string Reason_Internal                        { get; set; }       
        public Fortify_CodeLocation SourceLocation             { get; set; }       
        public Fortify_CodeLocation SecundaryLocation         { get; set; }
        public uint SourceLocation_ContextId                 { get; set; }
        public string SourceLocation_Snippet                 { get; set; }       
        public string SecundaryLocation_Snippet                { get; set; }               
        public string Label                                    { get; set; }               
       
        public Fortify_TraceEntry()
        {
            KnowledgeFacts = new List<Fortify_TraceEntryFact>();           
        }
    }
    public class Fortify_TraceEntryFact
    {
        public bool Primary {get;set;}
        public string Type {get;set;}
        public string Value {get;set;}
    }   
   
    public class Fortify_Vulnerability
    {
        public Fortify_Function Context { get; set; }    // from AnalysisInfo
        public Fortify_ReplacementDefinitions ReplacementDefinitions { get; set; }
        public List<Fortify_TraceEntry> Traces { get; set; }
        public string Kingdom { get; set; }                // from ClassInfo
        public string AnalyzerName { get; set; }
        public string ClassId { get; set; }
        public decimal DefaultSeverity { get; set; }
        public string  Type { get; set; }
        public string  SubType { get; set; }
       
        public decimal Confidence { get; set; }            // from InstanceInfo
        public string InstanceId { get; set; }
        public decimal InstanceSeverity { get; set; }                           
       
        public Fortify_Vulnerability()
        {
            ReplacementDefinitions = new Fortify_ReplacementDefinitions();
            Traces = new List<Fortify_TraceEntry>();
        }
    }

The main API_Fortify class is still quite small:

public class API_Fortify
    {               
        public Fortify_Scan convertToFortifyScan(string fvdlFile)
        {
            var scan = new Fortify_Scan();
            scan._fvdl = loadFvdl_Raw(fvdlFile);
            scan.mapFvdlData();
            return scan;
        }
       
        public FVDL loadFvdl_Raw(string fvdlFile)
        {
            "Loading fvdl file: {0}".info(fvdlFile);
            try
            {
                var chachedFvdl = (FVDL)O2LiveObjects.get(fvdlFile);
                if (chachedFvdl.notNull())
                    return chachedFvdl;
            }
            catch { }
            
            var o2Timer = new O2Timer("loading {0} file".format(fvdlFile.fileName())).start();       
             var _fvdl = FVDL.Load(fvdlFile);   
             O2LiveObjects.set(fvdlFile,_fvdl);
             o2Timer.stop();
             return _fvdl; 
        }
    }

With all the heavy lifting of convering the *.fvdl data into the multiple O2 Fortify_* classes , done via ExtensionMethods

public static class Fortify_Scan_ExtensionMethods_MappingFvdl
    {
        public static Fortify_Scan mapFvdlData(this Fortify_Scan fortifyScan)
        {
            var o2Timer = new O2Timer("Mapped Fvdl Data").start();
            fortifyScan.mapScanDetails()
                       .mapContextPool()
                       .mapDescriptions()
                       .mapCalledWithNoDefs()
                       .mapSinks()
                       .mapSources()
                       .mapSnippets()
                       .mapVulnerabilities();
            o2Timer.stop();          
            return fortifyScan;
        }
       
        public static Fortify_Scan mapScanDetails(this Fortify_Scan fortifyScan)
        {
            var fvdl = fortifyScan._fvdl;
            fortifyScan.BuildID = fvdl.Build.BuildID;
            fortifyScan.Loc = fvdl.Build.LOC;
            fortifyScan.SourceBasePath = fvdl.Build.SourceBasePath;
            fortifyScan.ScanTime = fvdl.Build.ScanTime.value;
            fortifyScan.CreatedDateTime = DateTime.Parse("{0} {1}".format(fortifyScan._fvdl.CreatedTS.date.ToShortDateString() ,
                                                                          fortifyScan._fvdl.CreatedTS.time.ToLongTimeString() ));                       
            fortifyScan.Errors = fvdl.EngineData.Errors.str()!= "<Errors xmlns=\"xmlns://www.fortifysoftware.com/schema/fvdl\" />"
                                        ? fvdl.EngineData.Errors.str()
                                        : "";
            foreach(var file in fvdl.Build.SourceFiles.File)
                fortifyScan.ScannedFiles.Add(new Fortify_ScannedFile ()
                                                    {
                                                        Loc = file.loc,
                                                        Size = file.size,
                                                        Timestamp = file.timestamp,
                                                        Type = file.type,
                                                        Path = file.TypedValue
                                                    });
            return fortifyScan;
        }
       
        public static Fortify_Scan mapContextPool(this Fortify_Scan fortifyScan)
        {
            foreach(var context in fortifyScan._fvdl.ContextPool.Context)
            {
                //var location = context.FunctionDeclarationSourceLocation;
                fortifyScan.Contexts.Add(
                    new Fortify_Context()
                            {
                                Id = context.id.str() ,   
                                Function = new Fortify_Function(context.Function.name,context.FunctionDeclarationSourceLocation)                                           
                            });                                       
            }
            return fortifyScan;       
        }               
       
        public static Fortify_Scan mapDescriptions(this Fortify_Scan fortifyScan)
        {                
                foreach(var description in fortifyScan._fvdl.Description)
                {
                      var fortifyDescription = new Fortify_Description
                              {
                                  Abstract = description.Abstract,
                                  ClassID = description.classID,
                                  ContentType = description.contentType,                                 
                                  Explanation = description.Explanation,
                                  Recommendations = description.Recommendations,
                            };
                    if (description.Tips.notNull())
                        foreach(var tip in description.Tips.Tip)
                            fortifyDescription.Tips.Add(tip);
                    fortifyScan.Descriptions.add(fortifyDescription);   
                }
                return fortifyScan;
        }
       
        public static Fortify_Scan mapCalledWithNoDefs(this Fortify_Scan fortifyScan)
        {
            //have to map this using the xElement because the xsd/cs schema doesn't support it (at the moment)  
            foreach(var function in fortifyScan._fvdl.ProgramData.CalledWithNoDef.xElement().elements())
            {               
                fortifyScan.CalledWithNoDefs.Add(
                    new Fortify_CalledWithNoDef()
                            {
                                Name = function.attribute("name").value(),
                                Namespace = function.attribute("namespace").value(),
                                EnclosingClass = function.attribute("enclosingClass").value()
                            });                                       
            }
            return fortifyScan;       
        }   
       
        public static Fortify_Scan mapSinks(this Fortify_Scan fortifyScan)
        {
            foreach(var sink in fortifyScan._fvdl.ProgramData.Sinks.SinkInstance)
            {               
                fortifyScan.Sinks.Add(
                    new Fortify_Sink()
                            {
                                RuleID = sink.ruleID.str() ,
                                Function_Call = new Fortify_Function(sink.FunctionCall.Function.name,sink.FunctionCall.SourceLocation)
                            });                                       
            }
            return fortifyScan;       
        }
       
        public static Fortify_Scan mapSources(this Fortify_Scan fortifyScan)
        {
            foreach(var source in fortifyScan._fvdl.ProgramData.Sources.SourceInstance)
            {               
                var fortifySource = new Fortify_Source()
                                            {
                                                RuleID = source.ruleID.str() ,                                               
                                            };
                if (source.FunctionCall.notNull())                                           
                    fortifySource.Function_Call = new Fortify_Function(source.FunctionCall.Function.name,source.FunctionCall.SourceLocation);
                if (source.FunctionCall.notNull())   
                    fortifySource.Function_Entry = new Fortify_Function(source.FunctionCall.Function.name,source.FunctionCall.SourceLocation);
                if (source.TaintFlags.notNull())
                    fortifySource.TaintFlags = (from taintFlag in source.TaintFlags.TaintFlag
                                                select taintFlag.name).toList();
                fortifyScan.Sources.Add(fortifySource);
            }
            return fortifyScan;       
        }
        public static Fortify_Scan mapSnippets(this Fortify_Scan fortifyScan)
        {
            foreach(var snippet in fortifyScan._fvdl.Snippets.Snippet)
            {               
                fortifyScan.Snippets.Add(
                    new Fortify_Snippet()
                            {
                                Id = snippet.id.str() ,   
                                Text = snippet.Text,
                                CodeLocation = new Fortify_CodeLocation()
                                                        {
                                                            Path = snippet.File,
                                                            Line = snippet.StartLine,
                                                            LineEnd = snippet.EndLine                                                           
                                                        }                                           
                            });                                       
            }           
            return fortifyScan;       
        }               

        public static Fortify_Scan mapVulnerabilities(this Fortify_Scan fortifyScan)
        {                
            foreach(var vulnerability in fortifyScan._fvdl.Vulnerabilities.Vulnerability)               
                if (vulnerability.notNull())
                {
                      var fortifyVulnerability = new Fortify_Vulnerability();
                           
                    //from ClassInfo                                       
                    fortifyVulnerability.AnalyzerName      = vulnerability.ClassInfo.AnalyzerName;
                    fortifyVulnerability.ClassId          = vulnerability.ClassInfo.ClassID;
                    fortifyVulnerability.DefaultSeverity = vulnerability.ClassInfo.DefaultSeverity;
                    fortifyVulnerability.Kingdom          = vulnerability.ClassInfo.Kingdom;                   
                    fortifyVulnerability.Type              = vulnerability.ClassInfo.Type;                   
                    fortifyVulnerability.SubType          = vulnerability.ClassInfo.Subtype;                   
                   
                    //from
                    fortifyVulnerability.InstanceId         = vulnerability.InstanceInfo.InstanceID;
                    fortifyVulnerability.InstanceSeverity     = vulnerability.InstanceInfo.InstanceSeverity;
                    fortifyVulnerability.Confidence         = vulnerability.InstanceInfo.Confidence;
                   
                    //
                   
                    //from AnalysisInfo       
                    var analysisInfo =     vulnerability.AnalysisInfo;                   
                    if (analysisInfo.Unified.notNull())
                    {
                        if (analysisInfo.Unified.Context.notNull() && analysisInfo.Unified.Context.Function.notNull())                       
                            fortifyVulnerability.Context = new Fortify_Function(analysisInfo.Unified.Context.Function.name,
                                                                                analysisInfo.Unified.Context.FunctionDeclarationSourceLocation);                       
                        if (analysisInfo.Unified.ReplacementDefinitions.notNull())                                                                       
                            foreach(var def in analysisInfo.Unified.ReplacementDefinitions.Def)                       
                                fortifyVulnerability.ReplacementDefinitions.Definitions.add(def.key, def.value);
                        foreach(var trace in analysisInfo.Unified.Trace)
                            foreach(var entry in trace.Primary.Entry)
                            {
                                var traceEntry = new Fortify_TraceEntry();
                                if (entry.NodeRef.notNull())
                                    traceEntry.NodeRefId = entry.NodeRef.id;
                                if (entry.Node.notNull())
                                {
                                    var node = entry.Node;
                                    traceEntry.DetailsOnly = node.detailsOnly ?? false;
                                    traceEntry.IsDefault = node.isDefault ?? false;
                                    traceEntry.Label = node.label ?? "";
                                   
                                    if (node.Action.notNull())
                                    {
                                        traceEntry.ActionType = node.Action.type;
                                        traceEntry.ActionValue = node.Action.TypedValue;
                                    }                                   
                                    if (node.Knowledge.notNull())
                                    {
                                        foreach(var fact in node.Knowledge.Fact)
                                            traceEntry.KnowledgeFacts.Add(new Fortify_TraceEntryFact()
                                                                                {
                                                                                    Primary = fact.primary,
                                                                                    Type = fact.type,
                                                                                    Value = fact.TypedValue
                                                                                });
                                    }                                   
                                    if (node.Reason.notNull())
                                    {
                                        traceEntry.Reason_RuleId = node.Reason.Rule.notNull()
                                                                        ? node.Reason.Rule.ruleID
                                                                        : "";
                                        traceEntry.Reason_TraceRef = node.Reason.TraceRef.notNull()
                                                                         ? node.Reason.TraceRef.str()
                                                                         : "";
                                        traceEntry.Reason_Internal = node.Reason.Internal.notNull()
                                                                         ? node.Reason.Internal.str()
                                                                         : "";                                    
                                    }   
                                    if (node.SourceLocation.notNull())
                                    {
                                        traceEntry.SourceLocation = new Fortify_CodeLocation(node.SourceLocation);
                                        traceEntry.SourceLocation_ContextId = node.SourceLocation.contextId ?? 0;
                                        traceEntry.SourceLocation_Snippet = "node.SourceLocation.snippet";
                                    }
                                    if (node.SecondaryLocation.notNull())
                                    {
                                        traceEntry.SecundaryLocation = new Fortify_CodeLocation(node.SecondaryLocation);                                       
                                        traceEntry.SecundaryLocation_Snippet = node.SecondaryLocation.snippet;                                       
                                    }                                   
                                }
                                fortifyVulnerability.Traces.Add(traceEntry);                                               
                            }
                    }       
                    fortifyScan.Vulnerabilities.add(fortifyVulnerability);   
                }
               
            return fortifyScan;
        }
    }       

 
Since all the data conversion is happening at the backend classes, the GUI script is now just focused on showing the data (with most objects shown on separate tabs)

//var topPanel = panel.clear().add_Panel(); 
var topPanel = "Util - FVDL viewer (drop *.fvdl on the left-hand-side PropertyGrid)".popupWindow(1000,500);
topPanel.insert_LogViewer();
  </pre>
&nbsp;

var tabControl = topPanel.add_TabControl();
var scannedFiles_Table = tabControl.add_Tab("Scanned Files").add_TableList();
var context_Table = tabControl.add_Tab("Contexts").add_TableList();
var description_Table = tabControl.add_Tab("Descriptions").add_TableList();
var calledWithNoDefs_Table = tabControl.add_Tab("CalledWithNoDefs").add_TableList();
var sinks_Table = tabControl.add_Tab("Sinks").add_TableList();
var sources_Table = tabControl.add_Tab("Sources").add_TableList();
var snippets_Table = tabControl.add_Tab("Snippets").add_TableList();
var vulnerabilities_Table = tabControl.add_Tab("Vulnerabilities").add_TableList();

//var traceNodes_Table = tabControl.add_Tab("TraceNodes").add_TableList();

var propertyGrid = topPanel.insert_Left().add_PropertyGrid();

Fortify_Scan fortifyScan = null;
var apiFortify = new API_Fortify();

Action<string> showUserMessage =
    (message)=> {
//                    userMessages_Label.set_Text(message);                     
                    message.info();
                };
               
Action<Fortify_Scan> showFvdl = 
    (scan) =>  
        {  
                 vulnerabilities_Table.title("Showing {0} Vulnerabilties".format(scan.Vulnerabilities.size()))   
                                      .show(scan.Vulnerabilities);        
                scannedFiles_Table.show(scan.ScannedFiles);
                context_Table.show(scan.Contexts); //.makeColumnWidthMatchCellWidth();
                description_Table.show(scan.Descriptions);
                calledWithNoDefs_Table.show(scan.CalledWithNoDefs);                
                sinks_Table.show(scan.Sinks);
                sources_Table.show(scan.Sources);
                vulnerabilities_Table.show(scan.Vulnerabilities);
                snippets_Table.show(scan.Snippets);
//                traceNodes_Table.show((from vulnerability in scan.Vulnerabilities    
//                                       select vulnerability.Traces).SelectMany(p=>p).toList()); 
        };
 
Action<string> loadAndShowFile =
    (file)=>{ 
                showUserMessage("... loading file: {0}".format(file.fileName())); 
                O2Thread.mtaThread(()=>{                                                                                          
                                            fortifyScan = apiFortify.convertToFortifyScan(file);
                                            showFvdl(fortifyScan);         
                                            propertyGrid.show(fortifyScan);
                                            showUserMessage("loaded fvdl file: {0}".format(file.fileName()));
                                        });
            };

propertyGrid.onDrop(loadAndShowFile);
  
return "done";       
      
//O2File:API_Fortify_1_6.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll 
 

July 18, 2011 Posted by | Fortify, Interoperability | Leave a comment

Fortify FVDL Files – First working Parser and Viewer for *.fvdl files

Following from the previous Fortify FVDL posts (herehere, here and here), here is a first working tool that is able to load up *.fvdl files, parse its relevant data into a new set of classes, and then visuzalize its contents into a number of tabs.

This is what the main GUI look like (after loading/dropping the 35Mb dspace.fvdl file):

On the left there is a PropertyGrid control that shows a view of the new O2 Fortify_Scan class, and on the right there is a TabControl with a number of Tab Pages (each containing a raw view of the classes created).

Here is what each of the  tab pages looks like:

ScannedFiles

Contexts:

Descriptions

CalledWithNoDefs

Sinks

Sources

Snippets

Vulnerabilities

 The next post will show what the updated API_Fortify looks like, and the code that was used to create this GUI

July 18, 2011 Posted by | Fortify, Interoperability | 1 Comment

Fortify FVDL files – Creating .NET classes that map to Fvdl xml structure

I’m still getting my head around how the *.fvdl files are structured, but after looking at the data that they seem to contains (using the fvdl.details() viewer), I’ve created the following classes which (I think) represent the type of data that is contained in the fvdl files (I tried to consolidate the data structures a bit, but I’m sure there is still quite a bit of refectoring and optimization that can be done):

    public class Fortify_Scan
    {
        public FVDL _fvdl;
       
        public string BuildID { get; set; }
        public string LOC { get; set; }
        public string SourceBasePath { get; set; }
        public string CreatedDate { get; set; }
        public string CreatedTime { get; set; }       
        public List<Fortify_Vulnerability> Vulnerabilities { get; set; }
        public List<Fortify_Context> Contexts { get; set; }
        public List<Fortify_Description> Descriptions { get; set; }
        public List<Fortify_Sink> Sinks { get; set; }
        public List<Fortify_Source> Sources { get; set; }
       
        public Fortify_Scan()
        {
            Vulnerabilities = new List<Fortify_Vulnerability>();
            Contexts = new List<Fortify_Context>();
            Descriptions = new List<Fortify_Description>();
            Sinks = new List<Fortify_Sink>();
            Sources = new List<Fortify_Source>();
        }
    }       
           
    public class Fortify_Context
    {
        public string Id { get; set; }
        public Fortify_Function Function { get; set; }
    }
   
    public class Fortify_Function
    {
        public string FunctionName { get; set; }
        public Fortify_CodeLocation CodeLocation { get; set; }       
    }
   
    public class Fortify_CodeLocation
    {
        public string Path { get; set; }
        public string Line { get; set; }
        public string LineEnd { get; set; }
        public string ColStart { get; set; }
        public string ColEnd { get; set; }
    }
   
    public class Fortify_Description
    {
        public string Abstract { get; set; }
        public string ClassID { get; set; }
        public string ContentType { get; set; }
        public string Explanation { get; set; }
        public string Recommendations { get; set; }
        public string Tips { get; set; }       
    }
   
    public class Fortify_Sink
    {
        public string ruleID { get; set; }
        public Fortify_Function Function_Call{ get; set;}               
    }
   
    public class Fortify_Source
    {
        public string ruleID { get; set; }
        public Fortify_Function Function_Call{ get; set;}               
        public Fortify_Function Function_Entry{ get; set;}               
        public List<string> TaintFlags { get; set;}
        public Fortify_Source()
        {
            TaintFlags = new List<string>();
        }
    }       
   
    public class Fortify_Snippet
    {
        public string Id { get; set; }
        public Fortify_CodeLocation CodeLocation { get; set; }       
        public string Text { get; set;}       
    }
    public class Fortify_TraceEntry
    {
        public Fortify_TraceEntryNode Node         { get; set; }
        public string NodeRef                     { get; set; }
    }
    public class Fortify_TraceEntryNode
    {
        public bool IsDefault                             { get; set; }
        public string ActionType                         { get; set; }
        public string ActionValue                         { get; set; }
        public Fortify_CodeLocation SourceLocation         { get; set; }
        public Fortify_CodeLocation SecundaryLocation     { get; set; }
        public Fortify_Snippet Snippet                     { get; set; }
        public string ContextId                            { get; set; }
        public string ReasonRuleId                        { get; set; }
        public string Label                                { get; set; }
        public List<Fortify_TraceEntryFact> Facts        { get; set; }
       
    }
   
    public class Fortify_TraceEntryFact
    {
        public bool Primary {get;set;}
        public string Type {get;set;}
        public string Value {get;set;}
    }
    public class Fortify_Vulnerability
    {
        public string Kingdom { get; set; }                // from ClassInfo
        public string Analyzer { get; set; }
        public string ClassId { get; set; }
        public decimal DefaultSeverity { get; set; }
        public string  Type { get; set; }
        public string  SubType { get; set; }
       
        public decimal Confidence { get; set; }            // from InstanceInfo
        public string InstanceId { get; set; }
        public decimal InstanceSeverity { get; set; }
       
       
        public Fortify_Function Function { get; set; }       
        public Items ReplacementDefinitions { get; set; }
        public List<Fortify_TraceEntry> Traces { get; set; }
    }

(also, the data structures that I’m seeing are directly mapped to the object that was created from the current XSD/C# file, so if you know the inner structure of the *.fvdl files and see missing bits of data that are very useful to have or visualize, please let me know)

To reflect the new Classes (and the fact that the main object is now the Fortify_Scan class), I’ve modified the API_Fortify class (note that there is still quite a bit to go, since the mapping functions are just importing some of the vulnerability data available)

    public class API_Fortify
    {               
        public Fortify_Scan convertToFortifyScan(string fvdlFile)
        {
            var scan = new Fortify_Scan();
            scan._fvdl = loadFvdl_Raw(fvdlFile);
            scan.mapFvdlData();
            return scan;
        }
       
        public FVDL loadFvdl_Raw(string fvdlFile)
        {
            try
            {
                var chachedFvdl = (FVDL)O2LiveObjects.get(fvdlFile);
                if (chachedFvdl.notNull())
                    return chachedFvdl;
            }
            catch { }
            
            var o2Timer = new O2Timer("loading {0} file".format(fvdlFile.fileName())).start();       
             var _fvdl = FVDL.Load(fvdlFile);   
             O2LiveObjects.set(fvdlFile,_fvdl);
             o2Timer.stop();
             return _fvdl; 
        }
    }
   
    public static class Fortify_Scan_ExtensionMethods_MappingFvdl
    {
        public static Fortify_Scan mapFvdlData(this Fortify_Scan fortifyScan)
        {
            fortifyScan.mapVulnerabilities();
            return fortifyScan;
        }
       
        public static Fortify_Scan mapVulnerabilities(this Fortify_Scan fortifyScan)
        {                
                foreach(var vulnerability in fortifyScan._fvdl.Vulnerabilities.Vulnerability)
                {
                      var fortifyVulnerability = new Fortify_Vulnerability
                              {
                                Kingdom = vulnerability.ClassInfo.Kingdom,
                                Analyzer = vulnerability.ClassInfo.AnalyzerName,
                                ClassId = vulnerability.ClassInfo.ClassID,
                                DefaultSeverity = vulnerability.ClassInfo.DefaultSeverity,
                                InstanceId = vulnerability.InstanceInfo.InstanceID,
                                InstanceSeverity = vulnerability.InstanceInfo.InstanceSeverity,
                                Confidence = vulnerability.InstanceInfo.Confidence,                                          
/*                                Function = vulnerability.AnalysisInfo.Unified.notNull() && vulnerability.AnalysisInfo.Unified.Context.Function.notNull()
                                            ? vulnerability.AnalysisInfo.Unified.Context.Function.name
                                            : "" ,
                                File = vulnerability.AnalysisInfo.Unified.notNull() && vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.notNull()
                                            ? vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.path
                                            : "" ,
                                Line = vulnerability.AnalysisInfo.Unified.notNull() && vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.notNull()
                                            ? vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.line
                                            : 0
*/                                           
                            };
                    fortifyScan.Vulnerabilities.add(fortifyVulnerability);   
                };                               
                return fortifyScan;
        }
    }       

Here is the updated viewer (that now consumes a Fortify_Scan object)

var topPanel = panel.clear().add_Panel();
//var topPanel = "Util - Simple FVDL viewer".popupWindow(1000,400);</pre>
&nbsp;

var tableList = topPanel.clear().add_TableList().title("Drop an *.fvdl file here to load it");
var propertyGrid = topPanel.insert_Left().add_PropertyGrid();
var apiFortify = new API_Fortify();

Action<List<Fortify_Vulnerability>> showFvdl =
    (vulnerabilities) =>
        {
                 tableList.title("Showing {0} Vulnerabilties".format(vulnerabilities.size()))
                         .show(vulnerabilities);                                                        
        };
 
Action<string> loadAndShowFile =
    (file)=>{
                tableList.title("... loading file: {0}".format(file.fileName()));
                O2Thread.mtaThread(()=>{                                                                                       
                                            var fortifyScan = apiFortify.convertToFortifyScan(file);
                                            showFvdl(fortifyScan.Vulnerabilities);       
                                            propertyGrid.show(fortifyScan);
                                        });
            };

tableList.onDrop(loadAndShowFile);
tableList.getListViewControl().onDrop(loadAndShowFile);
   
var testFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
loadAndShowFile(testFile);
/*(var _fortifyScan = apiFortify.convertToFortifyScan(xmlFile);
propertyGrid.show(_fortifyScan);   
showFvdl(_fortifyScan.Vulnerabilities);*/

return "done"; 

//O2File:C:\O2\O2Scripts_Database\_Scripts\3rdParty_Tools\Fortify\API_Fortify_1_6.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

Note that I added a PropertyGrid to the left of the GUI which will show the contents of the Fortity_Scan object

July 17, 2011 Posted by | Fortify, Interoperability | 2 Comments

Fortify FVDL files – Creating an API and consumining it

Following from the (Fortify FVDL related) Creating and consuming the schema and CSharp file and Simple TableList Viewer Tool posts, this one shows the next evolutionary step, which is the creation of an API that can be easily consumed by *.h2 scripts.

The script show here will have the same funcionality has the one shown in Simple TableList Viewer Tool, but the its structure will be completely different.

 The data will be stored in a dedicated class (previously we used an anonymous class)

public class Fortify_Vulnerability
    {
        public string Kingdom { get; set; }
        public string Analyzer { get; set; }
        public string ClassId { get; set; }
        public decimal DefaultSeverity { get; set; }
        public string InstanceId { get; set; }
        public decimal InstanceSeverity { get; set; }
        public decimal Confidence { get; set; }
        public string Function { get; set; }
        public string File { get; set; }
        public int Line { get; set; }
    }

And the main functions of loading and parsing are now exposed in an API file called API_Fortify_1_6.cs (which provides the class API_Fortify):

    public class API_Fortify
    {
        public FVDL loadFvdl(string fvdlFile)
        {
            try
            {
                var chachedFvdl = (FVDL)O2LiveObjects.get(fvdlFile);
                if (chachedFvdl.notNull())
                    return chachedFvdl;
            }
            catch { }
            
            var o2Timer = new O2Timer("loading {0} file".format(fvdlFile.fileName())).start();       
             var _fvdl = FVDL.Load(fvdlFile);   
             O2LiveObjects.set(fvdlFile,_fvdl);
             o2Timer.stop();
             return _fvdl; 
        }
       
       
        public List<Fortify_Vulnerability> getVulnerabilities(FVDL fvdl)
        {
                
                 var fortifyVulnerabities = new List<Fortify_Vulnerability>();               
                foreach(var vulnerability in fvdl.Vulnerabilities.Vulnerability)
                {
                      var fortifyVulnerability = new Fortify_Vulnerability
                              {
                                Kingdom = vulnerability.ClassInfo.Kingdom,
                                Analyzer = vulnerability.ClassInfo.AnalyzerName,
                                ClassId = vulnerability.ClassInfo.ClassID,
                                DefaultSeverity = vulnerability.ClassInfo.DefaultSeverity,
                                InstanceId = vulnerability.InstanceInfo.InstanceID,
                                InstanceSeverity = vulnerability.InstanceInfo.InstanceSeverity,
                                Confidence = vulnerability.InstanceInfo.Confidence,                                          
                                Function = vulnerability.AnalysisInfo.Unified.notNull() && vulnerability.AnalysisInfo.Unified.Context.Function.notNull()
                                            ? vulnerability.AnalysisInfo.Unified.Context.Function.name
                                            : "" ,
                                File = vulnerability.AnalysisInfo.Unified.notNull() && vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.notNull()
                                            ? vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.path
                                            : "" ,
                                Line = vulnerability.AnalysisInfo.Unified.notNull() && vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.notNull()
                                            ? vulnerability.AnalysisInfo.Unified.Context.FunctionDeclarationSourceLocation.line
                                            : 0
                            };
                    fortifyVulnerabities.add(fortifyVulnerability);   
                };
                return fortifyVulnerabities;
        }
    }       

The GUI script is now much smaller and is mailly focused on creating the GUI and consuming the API_Fortify class:

//var topPanel = panel.clear().add_Panel();
var topPanel = "Util - Simple FVDL viewer".popupWindow(1000,400);
var tableList = topPanel.clear().add_TableList().title("Drop an *.fvdl file here to load it");

var apiFortify = new API_Fortify();

Action<List<Fortify_Vulnerability>> showFvdl =
    (vulnerabilities) =>
        {
                 tableList.title("Showing {0} Vulnerabilties".format(vulnerabilities.size()))
                         .show(vulnerabilities);                                                        
        };
 
Action<string> loadAndShowFile =
    (file)=>{
                tableList.title("... loading file: {0}".format(file.fileName()));
                O2Thread.mtaThread(()=>{
                                            var fvdl = apiFortify.loadFvdl(file);
                                            showFvdl(apiFortify.getVulnerabilities(fvdl));
                                        });
            };

tableList.onDrop(loadAndShowFile);
tableList.getListViewControl().onDrop(loadAndShowFile);
   
//var xmlFile = @"C:\O2\_tempDir\_Fortify-Sate-2008\Fortify-Sate-2008\sate2008-Fvdl\naim.fvdl";
//var _fvdl = apiFortify.loadFvdl(xmlFile);
//showFvdl(apiFortify.getVulnerabilities(_fvdl));

return "done"; 

//O2File:C:\O2\O2Scripts_Database\_Scripts\3rdParty_Tools\Fortify\API_Fortify_1_6.cs
//O2Ref:O2_Misc_Microsoft_MPL_Libs.dll

The end result is the same as with the script shown in the  Simple TableList Viewer Tool post:

July 17, 2011 Posted by | Fortify, Interoperability | 1 Comment