OWASP O2 Platform Blog

Testing using GitGub hosted images for WordPress.com blog posts

One of the things that has been a massive pain when writing these blog posts has been the really bad workflow that WordPress.com has to deal with image’s uploads.

The workflow that I want is one where I can take a screenshot into my local clipboard and ‘automagically’ that image is available to be inserted in the blog (I’m happy to wait a coupe seconds as long as no other action is needed).

In the past, I did come up with a ‘semi automated way’ to upload these images via WordPressweb GUI (see O2 Script to automatically upload clipboard images to WordPress.com), but it was not 100% stable and recently that O2 script stopped working since WordPress.com changed its own image uploading code .

Since I’m now heavily into Git, and GitHub, I was thinking ‘why don’t I copy the image from the clipboard into a local Git Repository, then push that repository to the web, and expose the GitHub raw link available’

And with the latest version of O2 (in .NET 4.0 + native .NET Git suport) I was able to build such Gui.

Here it is in action:

 

After taking a screenshot, click on the ‘Commit…’ link and eventually the image will be shown in a WebBrowser and the image URL is also placed in the clipboard (for example https://github.com/DinisCruz/Images/raw/master/images/3_18_2012_6_00_48_AM.jpg)

Now to add an image to a blog post (like this one), one can use the url in the clipboard, or just drag and drop the image from the WebBrowser:

Here is the code that creates this GUI

var topPanel = O2Gui.open("Util - Copy Clipboard Image to GitHub Repository",400,400);
//var topPanel = panel.clear().add_Panel();
var actionsPanel = topPanel.insert_Above(40, "Actions");
var url_TextBox = topPanel.insert_Above(20).add_Label("Image Url").append_TextBox("").align_Right();
var browser = topPanel.add_WebBrowser_Control();
topPanel.insert_LogViewer();
var localRepository = @"C:\_WorkDir\O2\Images";
var nGit = new API_NGit();
nGit.open(localRepository);
actionsPanel.add_Link("Open Screenshot tool", ()=> API_Cropper.openGui())
            .append_Link("Commit image from clipboard and push",
                        ()=>{
                               
                                var image = localRepository.pathCombine("images").createDir().saveImageFromClipboardToFolder(); 
                                if (image.notNull())
                                {
                                    var o2Timer = new O2Timer("upload to Git").start();
                                    var url = "https://github.com/DinisCruz/Images/raw/master/images/{0}".format(image.fileName());
                                    "Current Status: {0}".info(nGit.status());
                                    nGit.add_and_commit_using_Status();                               
                                    nGit.push();
                                    "Url copied to Cliboard: {0}".info(url);
                                    url.clipboardText_Set();
                                    o2Timer.stop();
                                    browser.open(url);
                                }
                            });           



return nGit;
//O2File:API_NGit.cs
//O2File:API_Cropper.cs
//O2Ref:NGit.dll
//O2Ref:NSch.dll
//O2Ref:Mono.Security.dll
//O2Ref:Sharpen.dll

//O2Tag_DontAddExtraO2Files

March 18, 2012 Posted by | .NET, GitHub | Leave a comment

Decompressing HTTP GZIP responses using GZipStream (via extension methods)

Here is a good example of using Extension Methods to simplify the compressing and decompressing of strings.

This is how it is used:

var originalString = "this is the original string";

var compressedBytes = originalString.gzip_Compress();
var decompressedString = compressedBytes.gzip_Decompress();
return originalString == decompressedString; 
//O2File:_Extra_methods_Web.cs

Or even more simply:

return "this is a string".gzip_Compress().gzip_Decompress();
//O2File:_Extra_methods_Web.cs

Here is the source code of the gzip_Compress and gzip_Decompress Extension methods used above:

public static class _Extra_Web_ExtensionMethods_GZip
    {
        public static byte[] gzip_Compress(this string _string)
        {
            var bytes = Encoding.ASCII.GetBytes (_string);
            var outputStream = new MemoryStream();
            using (var gzipStream = new GZipStream(outputStream,CompressionMode.Compress))           
                gzipStream.Write(bytes, 0, bytes.size());           
            return outputStream.ToArray();
        }
       
        public static string gzip_Decompress(this byte[] bytes)
        {
            var inputStream = new MemoryStream();
            inputStream.Write(bytes, 0, bytes.Length);
            inputStream.Position = 0;
            var outputStream = new MemoryStream();
            using (var gzipStream= new GZipStream(inputStream,CompressionMode.Decompress))
            {
                byte[] buffer = new byte[4096];
                int numRead;
                while ((numRead = gzipStream.Read(buffer, 0, buffer.Length)) != 0)               
                    outputStream.Write(buffer, 0, numRead);               
            }   
            return outputStream.ToArray().ascii();
        }
    }

November 28, 2011 Posted by | .NET | Leave a comment

Weird Bug with RichTextBox’s ScrollToCaret while using AppendText (trying to fix text flickering)

While tuning the API_ConsoleOut (using a RichTextBox instead of a TextBox), I found a really weird behaviour with the RichTextBox’s ScrollToCaret method.

What happened is that since a call to AppendText didn’t show the last text added, a call to ScrollToCaret is needed. But, if multiple calls to AppendText + ScrollToCaret  are made the text view would be (alternatively) on the last line and the line before (this was aligned at the top, which meant that the text would flicker).

After multiple experiements (see O2 Script below) I found that if we set the HideSelection value to false, the AppendText will trigger the correct scroll.

Here is the code that works:

var topPanel = panel.clear().add_Panel();
topPanel.fill(false).height(50);
var richTextBox = topPanel.add_RichTextBox();
richTextBox.set_Text("a".line() +
                     "b".line() +
                     "c".line() +
                     "d".line());

richTextBox.hideSelection(false);
richTextBox.append_Text("123");
richTextBox.append_Text("_");
richTextBox.append_Text("*");     //all now work as expected

Here is the full code of my tests

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var topPanel = panel.clear().add_Panel();
topPanel.fill(false).height(50);
var richTextBox = topPanel.add_RichTextBox();
richTextBox.set_Text("a".line() +
                     "b".line() +
                     "c".line() +
                     "d".line());

//variation A2)
/*
richTextBox.append_Text("123").scrollToCaret();       // works as expeceted
richTextBox.append_Text("_").scrollToCaret();        // caret is one line up
//richTextBox.append_Text("*").scrollToCaret();        // works as expeceted
*/
//variation A1)                    
/*richTextBox.append_Text("123").scrollToCaret();    // works as expeceted
this.sleep(300);
richTextBox.append_Text("_").scrollToCaret();        // caret is one line up
this.sleep(300);
richTextBox.append_Text("*").scrollToCaret();         // works as expeceted
*/

//variation B)
/*richTextBox.invokeOnThread(
    ()=>{           
            richTextBox.AppendText("123");            // works as expeceted   
            richTextBox.ScrollToCaret();
            richTextBox.AppendText("_");            // caret is one line up
            richTextBox.ScrollToCaret();
            //richTextBox.AppendText("*");            // works as expeceted
            //richTextBox.ScrollToCaret();
        });*/
//variation C)
/*Action<string> appendText  =
    (text)=>{
                var before = richTextBox.SelectionStart;
                richTextBox.AppendText(text);
                var after = richTextBox.SelectionStart;
                "text: {0} before: {1}  after: {2}".info(text, before, after);               
                richTextBox.ScrollToCaret();                
            };
           
richTextBox.invokeOnThread(
    ()=>{                           
            appendText("123");  // works as expeceted
            appendText("_");    // caret is one line up
            appendText("*");  // works as expeceted
            appendText("_");    // caret is one line up
        });
*/
 
//the solution for the problem is to set the HideSelection property set to false
//which is now exposed via an O2 Extension Method

richTextBox.hideSelection(false); // richTextBox.showSelection(); also works
richTextBox.append_Text("123");
richTextBox.append_Text("_");
richTextBox.append_Text("*");     //all now work as expectted
/*
//Variation D) using scrollToCaret brings back the problem,
richTextBox.hideSelection(false);
richTextBox.append_Text("123");
richTextBox.append_Text("_");//.scrollToCaret();
richTextBox.append_Text("*").scrollToCaret(); 
*/
return "done";

It would be great to find out exactly what was causing this behaviour, but I looks that most of the RichTextBox behaviour is triggered by Windows Messages, so its quite a rabbit hole.

November 27, 2011 Posted by | .NET | Leave a comment

API_ConsoleOut.cs – New and InProcess Capture of the Console Output

There are multiple scenarious where it is usefull to capture (via Lambda callbacks) the calls to Console.Write. Most solutions out there are based on using the Process class (via its ProcessStartInfo settings):

The problem with those solutions is they are able to capture the Console Output of the new process created, NOT the current process (which is what I want since I need to see the Console messages created by an external API/Code that I consuming from O2)

Using O2 to captute Console Output of new processes

Note: If all you want to do is to capture the console output of a new process, there are a bunch of O2 methods/APIs that can help you with it:

Which can be used like this:

run cmd.exe (with params: /c dir) and get its console output

return "cmd.exe".startProcess_getConsoleOut(" /c dir");

run netstat.exe (with params: -na) and redirect its console into a lambda method

"netstat.exe".startProcess("-na", (line)=> line.info());

run netstat.exe and show its results in a table list

var topPanel = panel.clear().add_Panel();
var tableList = topPanel.add_TableList();
var lines = "netstat.exe".startProcess_getConsoleOut("-na")
                         .lines()
                         .removeRange(0,2);
var netstatData = (from line in lines
                   let fragments = line.split(" ").removeEmpty()
                   select new {
                                   @Type = fragments.value(0) , 
                                   Local_Address = fragments.value(1) ,
                                   Foreign_Address = fragments.value(2) ,
                                   State = fragments.value(3)
                                  });                  
tableList.show(netstatData);

Using O2 to captute Console Output in the current Process

To capture the Console Output in the current process, I first tried to capture the usage of a MemoryStream in a way simular to how Console actually works (see O2 blog post Dynamically creating private/internal types and delegates (UserCallBack and AsyncStreamReader) ) , but a much easier way is to specify a TextWriter via Console.SetOut.

To make it easier to use, I created the API_ConsoleOut class which can be used like this:

Show Console Ouput in a new window

"Write this to the console :)".open_ConsoleOut();
Console.Write("another");
Console.WriteLine("test");
//O2File:API_ConsoleOut.cs

Which looks like this:

 Show Console Output in an existing panel/control

var topPanel = panel.clear().add_Panel();
topPanel.show_ConsoleOut(); //this will open as a child of an existing control
Console.Write("another ");
Console.WriteLine("test");
topPanel.insert_Right().add_SourceCodeViewer().set_Text("<h1>some html<h1>",".html");
Console.WriteLine("now inside a Windows Control");

return "done";
//O2File:API_ConsoleOut.cs

Which looks like this:

 

Test GUI with Console Input

If you want to test the Console Input and Output, there is also a test GUI that you can use

var apiConsoleOut = new API_ConsoleOut();
apiConsoleOut.show_ConsoleOut_TestGui();

Console.Write("another ");
Console.WriteLine("test");
Console.WriteLine("now with more options");

//O2File:API_ConsoleOut.cs

Which looks like this:

Source code of API_Console_Out.cs

using System;
using System.IO;
using System.Text;
using System.Drawing;
using System.Windows.Forms;
using System.Collections.Generic;
using O2.Interfaces.O2Core;
using O2.Kernel;
using O2.Kernel.ExtensionMethods;
using O2.DotNetWrappers.ExtensionMethods;
using O2.Views.ASCX.classes.MainGUI;
using O2.XRules.Database.Utils;

//O2File:_Extra_methods_Misc.cs
//O2File:_Extra_methods_Windows.cs
//O2File:_Extra_methods_WinForms_Controls.cs

namespace O2.XRules.Database.APIs
{
    public class API_ConsoleOut : Control
    {            
        public O2StreamWriter o2StreamWriter;
       
        public API_ConsoleOut()
        {       
            this.o2StreamWriter = new O2StreamWriter();     
            this.o2StreamWriter.Name = "API_ConsoleOut";
            Console.SetOut(this.o2StreamWriter); 
        }
    }
   
    public class O2StreamWriter : TextWriter
    {
        public StringBuilder ConsoleOut     { get; set; }
        public StringBuilder CurrentLine    { get; set; }
        public List<string> Lines            { get; set; }
        public Action<char> On_NewChar        { get; set; }   
        public Action<string> On_NewLine     { get; set; }
        public bool LogAllChars                { get; set; }
        public bool LogAllLines                { get; set; }
        public string Name                    { get; set; }
        public int PauseBetweenCharWrite    { get; set; }
       
        public O2StreamWriter()
        {
            ConsoleOut = new StringBuilder();
            CurrentLine = new StringBuilder();
            Lines = new List<string>();       
            LogAllChars = false;
            LogAllLines = true;
            Name = "O2StreamWriter";
        }
   
        public override void Write(char value)
        {
            base.Write(value);
              this.ConsoleOut.Append(value.ToString());             
             
              if (value == '\n')
                  handleLine();                 
              else
                  this.CurrentLine.Append(value.ToString());
                 
              if (this.LogAllChars)
                "[{0}] char: {1}".info(this.Name, value);
           
            if (this.On_NewChar.notNull())
                this.On_NewChar(value);
           
            if (PauseBetweenCharWrite > 0)
                this.sleep(PauseBetweenCharWrite, false);
        }               
       
        public void handleLine()
        {
            var line = this.CurrentLine.str();
            if (line.lastChar('\r'))
                line = line.removeLastChar();
            this.CurrentLine = new StringBuilder();
            this.Lines.add(line);
           
            if (this.LogAllLines)
                "[{0}]: {1}".info(this.Name, line);
               
            if (this.On_NewLine.notNull())
                this.On_NewLine(line);           
        }
        public override Encoding Encoding
        {
            get { return System.Text.Encoding.UTF8; }
        }
    }
   
    public static class API_ConsoleOut_ExtensionMethods_TestGuis
    {
       
        public static API_ConsoleOut show_ConsoleOut_TestGui(this API_ConsoleOut apiConsoleOut)
        {
            var topPanel = O2Gui.open<Panel>("Console.Out capture Test",700,200);
            return apiConsoleOut.show_ConsoleOut_TestGui(topPanel);
        }
       
       
        public static API_ConsoleOut show_ConsoleOut_TestGui(this API_ConsoleOut apiConsoleOut, Control topPanel)
        {
            var consoleIn = topPanel.insert_Above(40, "ConsoleIn");
            var lines_TextBox = topPanel.clear().add_GroupBox("Console.Out Lines").add_TreeView();
            var chars_TextBox = topPanel.insert_Right("Console.Out Chars").add_TextArea();
                       
            consoleIn.add_TextBox("Write to send to Console Input ->","")
                     .onKeyPress_getChar( (_char)=> { if (_char == '\x0d') Console.WriteLine(); else  Console.Write(_char); })
                     .multiLine(true)
                     .focus();
                                
            apiConsoleOut.o2StreamWriter.On_NewChar = (_char) => chars_TextBox.append_Text(_char.str());
            apiConsoleOut.o2StreamWriter.On_NewLine = (line) => lines_TextBox.add_Node(line);
           
            //apiConsoleOut.o2StreamWriter.PauseBetweenCharWrite = 1;            
           
            Console.WriteLine("Welcome to show_ConsoleOut_TestGui");   
            Console.Write("Type something on the TextBox above to see it here".line());
            return apiConsoleOut;
        }
       
        public static API_ConsoleOut open_ConsoleOut(this string message)
        {
            return message.show_ConsoleOut();
        }
       
        public static API_ConsoleOut show_ConsoleOut(this string message)
        {
            var apiConsoleOut = new API_ConsoleOut();
            apiConsoleOut.show_ConsoleOut();
            Console.WriteLine(message);
            return apiConsoleOut;
        }
       
        public static API_ConsoleOut add_ConsoleOut(this Control topPanel)
        {
            return topPanel.show_ConsoleOut();
        }
       
        public static API_ConsoleOut show_ConsoleOut<T>(this T topPanel)
            where T : Control
        {
            return topPanel.show_ConsoleOut("Welcome to O2's console");
        }
       
        public static API_ConsoleOut show_ConsoleOut<T>(this T topPanel, string message )
            where T : Control
        {
            var apiConsoleOut = new API_ConsoleOut();
            apiConsoleOut.show_ConsoleOut(topPanel);
            Console.WriteLine(message);
            return apiConsoleOut;
        }
       
        public static API_ConsoleOut show_ConsoleOut(this API_ConsoleOut apiConsoleOut)
        {
            var topPanel = O2Gui.open<Panel>("Console.Out",700,200);
            return apiConsoleOut.show_ConsoleOut(topPanel);
        }
       
       
        public static API_ConsoleOut show_ConsoleOut(this API_ConsoleOut apiConsoleOut, Control topPanel)
        {
            var consoleOut = topPanel.add_TextArea();
            consoleOut.backColor(Color.Black);
            consoleOut.foreColor(Color.White);
            consoleOut.font("Lucida Console");
            apiConsoleOut.o2StreamWriter.On_NewChar = (_char) => consoleOut.append_Text(_char.str());
            apiConsoleOut.o2StreamWriter.LogAllLines = false;
           
            consoleOut.add_ContextMenu()
                      .add_MenuItem("Clear", false, ()=> consoleOut.set_Text(""))
                      .add_MenuItem("Write to Console Input", ()=> Console.WriteLine("Text to send to Console.Input".askUser()));
            return apiConsoleOut;
        }
    }
}

November 26, 2011 Posted by | .NET, Tools | 3 Comments

Dynamically creating private/internal types and delegates (UserCallBack and AsyncStreamReader)

When trying to create a callback into Console.Out, I tried to create a instance of the internal class AsyncStreamReader (which has a dependency on the private delegate UserCallBack). This workded ok  (see code below), but it didn’t have the expected behaviour (I did get a call into the outputReadNotifyUser Lambda method, but it only happened once and there was no value in its text variable) .

I actually found a better way (see next blog post), but meanwhile here is an  example of how to use reflection to create instance of private types and delegates.

//Not currently working, but shows how to create a private Delegate and Type

var currentProcess = Processes.getCurrentProcess();
Action<string> outputReadNotifyUser =
    (text)=>{
                "outputReadNotifyUser: {0}".info(text);
            };

var userCallBackType = PublicDI.reflection.getType("UserCallBack");
var userCallBack = Delegate.CreateDelegate(userCallBackType, outputReadNotifyUser, typeof(Action<string>).method("Invoke"));

var memoryStream = new MemoryStream();
var streamWriter = new StreamWriter(memoryStream);
System.Console.SetOut(streamWriter);
streamWriter.AutoFlush = true;

var oParams = new object[] { currentProcess,memoryStream, userCallBack , Encoding.UTF8};
var asyncStreamReader = PublicDI.reflection.getType("AsyncStreamReader");
var aSyncReader =  asyncStreamReader.ctors()[0].Invoke(oParams);
aSyncReader.invoke("BeginReadLine");

Console.Out.WriteLine("aaaa".line());
Console.Out.WriteLine("aaaa".line());

return "ok";
//using System.Text
//using System.IO
//using System.Diagnostics

November 25, 2011 Posted by | .NET, Reflection | 1 Comment

Good articles on the topic of “.Net DLL injection into another process”

I need to find a way to implement this technique so that O2 can be easily injected into any windows process 🙂

November 17, 2011 Posted by | .NET | Leave a comment

Creating the “Util – AspNet Control Encodings (Raw Format).h2” script

Here is the sequence that created the script described in Consuming ASP.NET Control Encoding mappings and visualizing them – Part 1 which visualizes the html based ASP.NET control encodings into an serializable C# class.

The ultimate objective is to create a complete/comprehensive ASP.NET Web Controls XSS Mappings object, and this is just the first step on that direction

Fetching Url contents, save as local file and show in IE

var sourceFile = "http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-components-postattachments/00-08-91-89-96/asp.net_5F00_control_5F00_encoding.htm";
var htmlContents = sourceFile.uri().getHtml();
var tempFile = htmlContents.saveAs( "".tempDir().pathCombine("asp.net_control_encodings.htm"));
//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);
return tempFile;

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

the url contents was saved to C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm

Viewing saved file

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);
return tempFile;

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

looking at the Html Elements of the IE Object

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);

ie.showElementsInTreeView();

return "ok";

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

Getting all rows (TR Element) – slow mode

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);

return ie.elements("TR");

return "ok";

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

Getting the first row  – slow mode

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);
ie.details();

var rows=  ie.elements("TR");
return rows[0];

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

Getting all rows – faster method

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);
ie.details();
return ie.IE.TableRows;

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

show rows data in treeView

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);
var rowsText = new List<string>();
foreach(var row in ie.IE.TableRows)
    rowsText.add(row.str());
var treeView = topPanel.insert_Left()
                       .add_TreeView()
                       .add_Nodes(rowsText);
return "ok";

//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

show rows data as a strongly type object in a table view: 

//var topPanel = O2Gui.open<Panel>("{name}",700,400);
var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);

var mappedData =  (from row in ie.IE.TableRows
                  select new {
                                 type = row.TableCells[0].str(),
                                 propertyName = row.TableCells[1].str(),
                                 attributeName_Script = row.TableCells[2].str(),
                                 htmlEncode_scriptEncode = row.TableCells[3].str(),
                                 urlEncode = row.TableCells[4].str()
                              }).toList();

topPanel.insert_Left()
        .add_TableList()
        .show(mappedData);

return     mappedData.save();   

//using WatiN.Core
//O2File:WatiN_IE_ExtensionMethods.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

Create a Dynamic type (won’t work due to serialization issues)

var dynamicTypeBuilder = "AspNetControlEncodings".dynamicType();
dynamicTypeBuilder.add_Properties<string>("type" , "propertyName", "attributeName_Script", "htmlEncode_scriptEncode","urlEncode");
var AspNetControlEncodings = dynamicTypeBuilder.create();
var encodingsList = AspNetControlEncodings.ctor().wrapOnList();

Populate data in a class that can be serialized

In the file AspNetControlEncodings.cs Create the class:

namespace O2.XRules.Database.Utils
{
    public class AspNetControlEncodings_Raw : List<AspNetControlEncoding_Raw>
    {   
                               
     }
   
    public class AspNetControlEncoding_Raw
    {   
        [XmlAttribute] public string @Type                         { get; set;}
        [XmlAttribute] public string PropertyName                { get; set;}
        [XmlAttribute] public string AttributeName_Script        { get; set;}
        [XmlAttribute] public string HtmlEncode_scriptEncode    { get; set;}
        [XmlAttribute] public string UrlEncode                    { get; set;}
    }
   
}

The AspNetControlEncodings.cs is now used/consumed by this script

//var topPanel = O2Gui.open<Panel>("{name}",700,400);

var tempFile = @"C:\O2\_tempDir\11-8-2011\asp.net_control_encodings.htm";
var topPanel = panel.clear().add_Panel();
var ie = topPanel.add_IE_with_NavigationBar().silent(false); 
ie.open(tempFile);

var mappedData =  (from row in ie.IE.TableRows
                  select new AspNetControlEncoding_Raw {
                                 @Type = row.TableCells[0].str(),
                                 PropertyName = row.TableCells[1].str(),
                                 AttributeName_Script = row.TableCells[2].str(),
                                 HtmlEncode_scriptEncode = row.TableCells[3].str(),
                                 UrlEncode = row.TableCells[4].str()
                              }).toList();

topPanel.insert_Left()
        .add_TableList()
        .show(mappedData);

return     mappedData.save();   

return "ok";
//using WatiN.Core
//O2File:WatiN_IE_ExtensionMethods.cs
//O2File:AspNetControlEncodings.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

The data will be shown in a table list and a temp file will be created. In this case C:\O2\_tempDir\11-9-2011\tmp3A1F.tmp.xml (which will be save as AspNetControlEncodings_Raw.xml in the O2 Scripts folder)

show data in TableList directly (must faster)

var topPanel = panel.clear().add_Panel();
var tempFile = @"C:\O2\_tempDir\11-9-2011\tmp3A1F.tmp.xml";

var mappedData = tempFile.load<AspNetControlEncodings_Raw>();

topPanel.add_TableList()
        .show(mappedData);

return     mappedData.save();   
//using WatiN.Core
//O2File:WatiN_IE_ExtensionMethods.cs
//O2File:AspNetControlEncodings.cs
//using O2.XRules.Database.Utils.O2
//O2Ref:WatiN.Core.1x.dll

Show data in popup form

var topPanel = O2Gui.open<Panel>("Util - AspNet ControlEncodings (Raw Format) ",700,400);

var mappedData  = "AspNetControlEncodings_Raw.xml".local().load<AspNetControlEncodings_Raw>();
topPanel.add_TableList()
        .show(mappedData);

//O2File:AspNetControlEncodings.cs

FinalScript:

ColorCoding data in popupForm (this is the Final version of the Script)

//var topPanel = panel.clear().add_Panel();
var topPanel = O2Gui.open<Panel>("Util - AspNet ControlEncodings (Raw Format)",700,400);

var mappedData  = "AspNetControlEncodings_Raw.xml".local().load<AspNetControlEncodings_Raw>();
var tableList = topPanel.add_TableList()
                        .show(mappedData);
tableList.add_Column("vuln");   
                   
tableList.visible(false);                       
foreach(var row in tableList.rows())
{
    var values = row.values();
    if (values[2] == "na")
        row.textColor(Color.Black); 
    else if (values[3].toBool() && values[4].toBool())
        row.textColor(Color.DarkGreen);
    else if (values[3].toBool() || values[4].toBool())
        row.textColor(Color.DarkOrange);
    else
    {
        row.textColor(Color.Red);                       
        row.SubItems[1].Text = "* " + values[1];    
    }
}
tableList.visible(true);
return "ok";
//O2File:AspNetControlEncodings.cs

November 16, 2011 Posted by | .NET, ASP.NET Controls, Fixing Code | 1 Comment

Consuming ASP.NET Control Encoding mappings and visualizing them – Part 1

Trying to find out the exact behaviour of the default ASP.NET Controls enconding behaviour is not simple, and surprisingly there are no good sources out there that clearly show what happens.

The best I could find is the table from Sasha Faust Which ASP.NET Controls Automatically Encodes?  blog post . Look in the attached asp.net_control_encoding.htm file, which looks like this:

The problem with that file is that is it not in a consumable format (it is in an html table).

So the first thing we need to do is to extract this data into a serializable .NET C# class which can be further analyzed.

To do that I wrote this script Util – AspNet ControlEncodings (Raw Format).h2 which looks like this when executed:

What is nice from that table is that due to the color coding and small change on the PropertyName value, we can now see:

all mappings that do no Encoding:

the mappings that have at least one of Encodings Enabled (either HtmlEncode or HtmlEncode):

the mappings that have both encodings enabled:

the mappings that don’t have a AttributeName defined:

Data stored in XML

These views are created from a serialized xml object (on AspNetControlEncodings.cs) which is stored in serialized format as an XML file:

Source Code

Here is the source code of this script:

//var topPanel = panel.clear().add_Panel();
var topPanel = O2Gui.open<Panel>("Util - AspNet ControlEncodings (Raw Format)",700,400);

var mappedData  = "AspNetControlEncodings_Raw.xml".local().load<AspNetControlEncodings_Raw>();
var tableList = topPanel.add_TableList()
                        .show(mappedData);
tableList.add_Column("vuln");   
                   
tableList.visible(false);                       
foreach(var row in tableList.rows())
{
    var values = row.values();
    if (values[2] == "na")
        row.textColor(Color.Black); 
    else if (values[3].toBool() && values[4].toBool())
        row.textColor(Color.DarkGreen);
    else if (values[3].toBool() || values[4].toBool())
        row.textColor(Color.DarkOrange);
    else
    {
        row.textColor(Color.Red);                       
        row.SubItems[1].Text = "* " + values[1];    
    }
}
tableList.visible(true);
return "ok";

//O2File:AspNetControlEncodings.cs

How the AspNetControlEncodings_Raw.xml file was created

To see how this script was created and how the original html table was transformed into the xml file, see this blog post: Creating the “Util – AspNet Control Encodings (Raw Format).h2″ script

November 16, 2011 Posted by | .NET, ASP.NET Controls | 1 Comment

O2 Script: AntiXSS – Test multiple Encodings

On the topic of AntiXSS here is a script I wrote ages ago called “AntiXSS – Test multiple Encodings.h2” which shows quickly the different behaviours of the different .NET encoding APIs:

Here is the source of this script:

var topPanel = O2Gui.open<Panel>("AntiXSS and HttpUtility encodings", 600,300); 
var result = topPanel.add_TextArea();
Action<string> showEncodings =
    (textToEncode)=>{
                        result.set_Text("");
                        result.append_Line("AntiXss.HtmlEncode -> {0}".format(AntiXss.HtmlEncode(textToEncode)));
                        result.append_Line("------------------------");
                        result.append_Line("AntiXss.UrlEncode -> {0}".format(AntiXss.UrlEncode(textToEncode)));
                        result.append_Line("------------------------");
                        result.append_Line("AntiXss.JavascriptEncode -> {0}".format(AntiXss.JavaScriptEncode(textToEncode)));
                        result.append_Line("------------------------");
                        result.append_Line("System.Web.HttpUtility.HtmlEncode -> {0}".format(System.Web.HttpUtility.HtmlEncode(textToEncode)));
                        result.append_Line("------------------------");
                        result.append_Line("System.Web.HttpUtility.UrlEncode -> {0}".format(System.Web.HttpUtility.UrlEncode(textToEncode)));
                        result.append_Line("------------------------");
                        result.append_Line("Original string (unencoded) -> {0}".format(textToEncode));
                        result.append_Line("------------------------");
                    };

var testPayload = "abc 123 \" ' < > \n :   ;   ".line() + "After an Enter";                    
result.insert_Above<Panel>(20)
      .add_LabelAndTextAndButton("Payload", testPayload, "convert", showEncodings);

showEncodings(testPayload);

//return AntiXss.HtmlEncode(payload);
//return "AntiXSSLibrary.dll".assembly().methods();
//using Microsoft.Security.Application
//O2Ref:AntiXSSLibrary.dll

November 16, 2011 Posted by | .NET, AntiXss | 2 Comments

O2 Script with Web Encoder and Decoder (with AntiXss Support)

A couple days ago I needed to do a number of Encodings/Decodings  in sequence (Encoded Text -> UrlDecode -> UrlDecode-> HtmlDecode), and since there was no easy way to do that automatically with other tools, I wrote the “Util – Web Encoder (with AntiXss Support).h2” script which looks like this:

Here is the method that runs the transformation (and show what is currently supported)

Func<string,string, string> applyTransformation =
    (type, text)=>{
                    if (type.valid() && text.valid() )
                    {
                        switch(type)
                        {
                            case "none":                                break;
                            case "HtmlDecode":                            return text.htmlDecode();
                            case "HtmlEncode":                            return text.htmlEncode();
                            case "UrlDecode":                            return text.urlDecode();
                            case "UrlEncode":                            return text.urlEncode();       
                           
                            case "AntiXss.HtmlEncode":                     return Encoder.HtmlEncode(text);
                            case "AntiXss.UrlEncode":                     return Encoder.UrlEncode(text);
                            case "AntiXss.JavaScriptEncode":             return Encoder.JavaScriptEncode(text);   
                            case "AntiXss.CssEncode":                     return Encoder.CssEncode(text);   
                            case "AntiXss.HtmlAttributeEncode":         return Encoder.HtmlAttributeEncode(text);   
                            case "AntiXss.HtmlFormUrlEncode":             return Encoder.HtmlFormUrlEncode(text);   
                            case "AntiXss.XmlAttributeEncode":             return Encoder.XmlAttributeEncode(text);   
                            case "AntiXss.XmlEncode":                     return Encoder.XmlEncode(text);   
                            case "AntiXss.VisualBasicScriptEncode":     return Encoder.VisualBasicScriptEncode(text);   
                            case "AntiXss.LdapDistinguishedNameEncode": return Encoder.LdapDistinguishedNameEncode(text);   
                            case "AntiXss.LdapFilterEncode":             return Encoder.LdapFilterEncode(text);   
                           
                            case "Sanitizer.GetSafeHtml":                return Sanitizer.GetSafeHtml(text);
                            case "Sanitizer.GetSafeHtmlFragment":        return Sanitizer.GetSafeHtmlFragment(text);
                           
                            default:
                                return text + "  not supported: {0}".format(type);
                        }                   
                    }
                    return text;
                  };

This uses the latest version of the AntiXSS library, including the new HtmlSanitizationLibrary.dll which has the GetSafeHtml* methods and looks really powerful.

Here is the entire code of this script:


var topPanel = O2Gui.open<Panel>("Util - Web Encoder (with AntiXss Support)",1000,400);
//var topPanel = panel.clear().add_Panel();
var configPanel = topPanel.insert_Above(40,"Config");
var sourceText = topPanel.add_GroupBox("Source Text").add_SourceCodeViewer(); 
var transformedText = topPanel.insert_Right("Transformed Text").add_SourceCodeViewer();
 
var transformationsAvailable = new List<string> { "none",
                                                  "HtmlDecode",    "HtmlEncode","UrlDecode", "UrlEncode",
                                                  "AntiXss.HtmlEncode",              "AntiXss.UrlEncode",                      "AntiXss.JavaScriptEncode",      "AntiXss.CssEncode",
                                                  "AntiXss.HtmlAttributeEncode",    "AntiXss.HtmlFormUrlEncode",             "AntiXss.XmlAttributeEncode",    "AntiXss.XmlEncode",
                                                  "AntiXss.VisualBasicScriptEncode","AntiXss.LdapDistinguishedNameEncode",    "AntiXss.LdapFilterEncode",
                                                  "Sanitizer.GetSafeHtml",             "Sanitizer.GetSafeHtmlFragment"};
var transformMode_1 = "";
var transformMode_2 = "";
var transformMode_3 = "";
Func<string,string, string> applyTransformation =
    (type, text)=>{
                    if (type.valid() && text.valid() )
                    {
                        switch(type)
                        {
                            case "none":                                break;
                            case "HtmlDecode":                            return text.htmlDecode();
                            case "HtmlEncode":                            return text.htmlEncode();
                            case "UrlDecode":                            return text.urlDecode();
                            case "UrlEncode":                            return text.urlEncode();       
                           
                            case "AntiXss.HtmlEncode":                     return Encoder.HtmlEncode(text);
                            case "AntiXss.UrlEncode":                     return Encoder.UrlEncode(text);
                            case "AntiXss.JavaScriptEncode":             return Encoder.JavaScriptEncode(text);   
                            case "AntiXss.CssEncode":                     return Encoder.CssEncode(text);   
                            case "AntiXss.HtmlAttributeEncode":         return Encoder.HtmlAttributeEncode(text);   
                            case "AntiXss.HtmlFormUrlEncode":             return Encoder.HtmlFormUrlEncode(text);   
                            case "AntiXss.XmlAttributeEncode":             return Encoder.XmlAttributeEncode(text);   
                            case "AntiXss.XmlEncode":                     return Encoder.XmlEncode(text);   
                            case "AntiXss.VisualBasicScriptEncode":     return Encoder.VisualBasicScriptEncode(text);   
                            case "AntiXss.LdapDistinguishedNameEncode": return Encoder.LdapDistinguishedNameEncode(text);   
                            case "AntiXss.LdapFilterEncode":             return Encoder.LdapFilterEncode(text);   
                           
                            case "Sanitizer.GetSafeHtml":                return Sanitizer.GetSafeHtml(text);
                            case "Sanitizer.GetSafeHtmlFragment":        return Sanitizer.GetSafeHtmlFragment(text);
                           
                            default:
                                return text + "  not supported: {0}".format(type);
                        }                   
                    }
                    return text;
                  };
                 
                 
Action applyTransformations =
    ()=>{
            var originalText = sourceText.get_Text();
            var result = applyTransformation(transformMode_1,originalText);
            result = applyTransformation(transformMode_2, result);
            result = applyTransformation(transformMode_3, result);
            transformedText.set_Text(result);            
        };

sourceText.onTextChange(
    (text)=>{    
                applyTransformations();
            }); 
 
configPanel.add_Label("Color Code the text as").top(3)
           .append_Control<ComboBox>().dropDownList().top(0)
                                         .add_Items(".xml",".html",",cs")                                        
                                         .onSelection((value)=> {
                                                                     transformedText.editor().setDocumentHighlightingStrategy(value.str());
                                                                     sourceText.editor().setDocumentHighlightingStrategy(value.str());
                                                                 })
                                         .selectFirst()
                                        
            .append_Label("Transform using:").top(3).autoSize() 
            .append_Control<ComboBox>().dropDownList().top(0).width(170).comboBoxHeight(250)
                                         .add_Items(transformationsAvailable)
                                         .onSelection<string>((value)=> { transformMode_1 = value; applyTransformations(); } )
                                         .selectFirst()
                                        
            .append_Label("and:").top(3).autoSize()
            .append_Control<ComboBox>().dropDownList().top(0).width(170).comboBoxHeight(250)
                                         .add_Items(transformationsAvailable)
                                         .onSelection<string>((value)=> { transformMode_2 = value; applyTransformations(); } )
                                        
            .append_Label("and:").top(3).autoSize() 
            .append_Control<ComboBox>().dropDownList().top(0).width(170).comboBoxHeight(250)
                                         .add_Items(transformationsAvailable)
                                         .onSelection<string>((value)=> { transformMode_3 = value; applyTransformations(); } )
                                         ;
 
sourceText.set_Text("this is a <b> test </b>");

return "done";
//using Microsoft.Security.Application
//O2Ref:AntiXSSLibrary.dll
//O2Ref:HtmlSanitizationLibrary.dll

November 16, 2011 Posted by | .NET, AntiXss, Fixing Code | Leave a comment