Category: Dev

Typosquatting generator

From Wikipedia: “Typosquatting, also called URL hijacking, a sting site, or a fake URL, is a form of cybersquatting, and possibly brandjacking which relies on mistakes such as typos made by Internet users when inputting a website address into a web browser. Should a user accidentally enter an incorrect website address, they may be led to any URL (including an alternative website owned by a cybersquatter).”

This C# code allow you to create a list of typos starting from a string. 

using System;
using System.Collections.Generic;
using System.Linq;

namespace TypoSample
{
    class Program
    {
        static void Main(string[] args)
        {
            List<string> strings = new Typo().GetList("emiliano");

            foreach (string s in strings)
            {
                Console.WriteLine(s);
            }
        }
    }

    internal class Typo
    {
        private static List<Character> characters = new List<Character>()
        {
            new Character() { Value = 'a', Similar = new List<char>() { '4' } },
            new Character() { Value = 'b', Similar = new List<char>() { } },
            new Character() { Value = 'c', Similar = new List<char>() {  } },
            new Character() { Value = 'd', Similar = new List<char>() {  } },
            new Character() { Value = 'e', Similar = new List<char>() { '3' } },
            new Character() { Value = 'f', Similar = new List<char>() {  } },
            new Character() { Value = 'g', Similar = new List<char>() { '6' } },
            new Character() { Value = 'h', Similar = new List<char>() { } },
            new Character() { Value = 'i', Similar = new List<char>() { '1', 'l' } },
            new Character() { Value = 'j', Similar = new List<char>() { } },
            new Character() { Value = 'k', Similar = new List<char>() { } },
            new Character() { Value = 'l', Similar = new List<char>() { '1', 'i' } },
            new Character() { Value = 'm', Similar = new List<char>() { 'n' } },
            new Character() { Value = 'n', Similar = new List<char>() { 'm' } },
            new Character() { Value = 'o', Similar = new List<char>() { '0' } },
            new Character() { Value = 'p', Similar = new List<char>() { 'q' } },
            new Character() { Value = 'q', Similar = new List<char>() { 'p' } },
            new Character() { Value = 'r', Similar = new List<char>() {  } },
            new Character() { Value = 's', Similar = new List<char>() { '5' } },
            new Character() { Value = 't', Similar = new List<char>() { 'f' } },
            new Character() { Value = 'u', Similar = new List<char>() { 'v' } },
            new Character() { Value = 'v', Similar = new List<char>() { 'u' } },
            new Character() { Value = 'w', Similar = new List<char>() {  } },
            new Character() { Value = 'x', Similar = new List<char>() {  } },
            new Character() { Value = 'y', Similar = new List<char>() {  } },
            new Character() { Value = 'z', Similar = new List<char>() { 's' } },
            new Character() { Value = '0', Similar = new List<char>() { 'o' } },
            new Character() { Value = '1', Similar = new List<char>() { 'i', 'l' } },
            new Character() { Value = '2', Similar = new List<char>() {  } },
            new Character() { Value = '3', Similar = new List<char>() { 'e' } },
            new Character() { Value = '4', Similar = new List<char>() { 'a' } },
            new Character() { Value = '5', Similar = new List<char>() { 's' } },
            new Character() { Value = '6', Similar = new List<char>() {  } },
            new Character() { Value = '7', Similar = new List<char>() {  } },
            new Character() { Value = '8', Similar = new List<char>() {  } },
            new Character() { Value = '9', Similar = new List<char>() {  } },
        };

        private List<string> calculated = new List<string>();

        private void Process(string s, string original)
        {
            char[] ca = s.ToCharArray();

            foreach (char c in ca)
            {
                Character cs = characters.Where(x => x.Value == c).FirstOrDefault();

                foreach (char ch in cs.Similar)
                {
                    string f = s.Replace(c, ch);

                    if (!calculated.Contains(f) && f != original)
                    {
                        calculated.Add(f);

                        Process(f, original);
                    }

                }
            }
        }

        internal List<string> GetList(string text)
        {
            Process(text.ToLower(), text.ToLower());

            return this.calculated;
        }
    }

    internal class Character
    {
        internal Character()
        {
            this.Similar = new List<char>();
        }

        internal char Value { get; set; }
        internal List<char> Similar { get; set; }
    }
}

Executing this program with the string “emiliano” you get this list of output strings

3miliano
3niliano
eniliano
emiliamo
3miliamo
3m1l1amo
em1l1amo
en1l1ano
3n1l1ano
3nlllano
enlllano
emlllamo
3mlllamo
3m111amo
em111amo
en111ano
3n111ano
3niiiano
eniiiano
emiiiamo
3miiiamo
3miii4mo
emiii4mo
eniii4no
3niii4no
3n1114no
en1114no
em1114mo
3m1114mo
3mlll4mo
emlll4mo
enlll4no
3nlll4no
3nlll4n0
enlll4n0
emlll4m0
3mlll4m0
3m1114m0
em1114m0
en1114n0
3n1114n0
3niii4n0
eniii4n0
emiii4m0
3miii4m0
3miiiam0
emiiiam0
eniiian0
3niiian0
3n111an0
en111an0
em111am0
3m111am0
3mlllam0
emlllam0
enlllan0
3nlllan0
3n1i1ano
en1i1ano
em1i1amo
3m1i1amo
3mlilamo
emlilamo
enlilano
3nlilano
3nl1lano
enl1lano
eml1lamo
3ml1lamo
3mi1iamo
emi1iamo
eni1iano
3ni1iano
3ni1i4no
eni1i4no
emi1i4mo
3mi1i4mo
3ml1l4mo
eml1l4mo
enl1l4no
3nl1l4no
3nlil4no
enlil4no
emlil4mo
3mlil4mo
3m1i14mo
em1i14mo
en1i14no
3n1i14no
3n1l14no
en1l14no
em1l14mo
3m1l14mo
3mili4mo
emili4mo
enili4no
3nili4no
3nili4n0
enili4n0
emili4m0
3mili4m0
3m1l14m0
em1l14m0
en1l14n0
3n1l14n0
3n1i14n0
en1i14n0
em1i14m0
3m1i14m0
3mlil4m0
emlil4m0
enlil4n0
3nlil4n0
3nl1l4n0
enl1l4n0
eml1l4m0
3ml1l4m0
3mi1i4m0
emi1i4m0
eni1i4n0
3ni1i4n0
3ni1ian0
eni1ian0
emi1iam0
3mi1iam0
3ml1lam0
eml1lam0
enl1lan0
3nl1lan0
3nlilan0
enlilan0
emlilam0
3mlilam0
3m1i1am0
em1i1am0
en1i1an0
3n1i1an0
3n1l1an0
en1l1an0
em1l1am0
3m1l1am0
3miliam0
emiliam0
enilian0
3nilian0
3m1l1ano
em1l1ano
emlllano
3mlllano
3m111ano
em111ano
emiiiano
3miiiano
3miii4no
emiii4no
em1114no
3m1114no
3mlll4no
emlll4no
emlll4n0
3mlll4n0
3m1114n0
em1114n0
emiii4n0
3miii4n0
3miiian0
emiiian0
em111an0
3m111an0
3mlllan0
emlllan0
em1i1ano
3m1i1ano
3mlilano
emlilano
eml1lano
3ml1lano
3mi1iano
emi1iano
emi1i4no
3mi1i4no
3ml1l4no
eml1l4no
emlil4no
3mlil4no
3m1i14no
em1i14no
em1l14no
3m1l14no
3mili4no
emili4no
emili4n0
3mili4n0
3m1l14n0
em1l14n0
em1i14n0
3m1i14n0
3mlil4n0
emlil4n0
eml1l4n0
3ml1l4n0
3mi1i4n0
emi1i4n0
emi1ian0
3mi1ian0
3ml1lan0
eml1lan0
emlilan0
3mlilan0
3m1i1an0
em1i1an0
em1l1an0
3m1l1an0
3milian0
emilian0

To modify the behavior of the program you can easily modify the “characters” list

The Smith Project

The project

The project starts from the desire to monitor the Internet in search of threats but also in search of situations that are not correlated with each other but which, with time or with support, may be at the basis of larger and currently unpredictable phenomena.

The first phase of the project deals with timely monitoring: the solution monitors the domains that are registered, collects information on registrations and hosting and checks the contents of the sites. This step allows you to quickly identify the different types of cyber threats. The collected data can be used for investigations and analyzes.

When a threat (present or probable) is identified, this is reported to security companies who send it to the specific blacklist and then a tweet is produced which is published on my profile: https://twitter.com/ecarlesi

The analysis of the data collected during this phase can be used as a history to identify patterns that allow forecasts on future scenarios.

The second phase of the project aims to produce evidence of phenomena deriving them from patterns discovered by the analysis performed by the components operating in phase one.

Currently, approximately 250000-300000 second-level domains are registered every day. Many of these domains are used to carry out cyber threats: spam, phishing, c2c, etc.

The information that can be acquired through the WHOIS service is not really useful in most cases. In fact, due to the anonymization options, the data are too generic and do not allow to be traced back to the real owner of the domain.

The only fact that is currently taken into account by the solution is the company that registered.

Not all providers have the same reputation. Users who make massive registrations, for example, tend to use cheaper providers who therefore see their reputation lower than others and consequently we attribute a lower initial score to registrations made with these companies.

Another indicator that is taken into consideration by the solution is that linked to the SSL certificate, its issuer and its duration.

This first information collected contributes to the production of a score that is associated with each domain. This score is added to that produced by the subsequent phases and thus contributes to the overall evaluation of the domain.

The main analysis phase is the one where the contents of the website are analyzed. The contents are downloaded and analyzed to verify that there is clearly dangerous or potentially dangerous content. The verification of the contents is based on a database of signatures which is enriched daily and which in the future will be able to learn from the analysis history.

The solution is based on several underlying systems which interacting allow to implement the required logic. The following paragraphs describe the main systems and their roles.

Zefiro

The Zefiro project was born from the idea of having active monitoring of the domains that are registered. This monitoring allows you to “see what happens in the world” before this actually happens (the purchase of the Internet domain, in fact, turns out to be one of the first activities that are carried out when starting a project). The project fulfills this requirement: to receive notification of domains that are registered in a short time, on average a few (10-16) hours.

This component was developed using .NET Framework 4.7 and runs on Windows 2019 using a SQL Server 2019 database. The evolution of this project will be in its rewrite using .NET Core.

Currently the component writes its logs into files. In the future, these files will have to flow into a logs management platform to allow more immediate monitoring and the creation of alerts in the face of logged events.

Miniluv

This project uses information related to the domain (domain name, WHOIS, HTTPS certificate and more) to associate a risk score to each domain name. This score is used later to alter the normal monitoring mode. The domains with a score below a certain threshold are sent via a notification to the subscribers of a specific mailing list.

This component was developed using .NET Framework 4.7 and runs on Windows 2019 using a SQL Server 2019 database. The evolution of this project will be in its rewrite using .NET Core.

Currently the component writes its logs into files. In the future, these files will have to flow into a logs management platform to allow more immediate monitoring and the creation of alerts in the face of logged events.

Smith

The Smith project implements the logic of monitoring and orchestrates the agents who perform the scans. Each scan produces a report which is saved by Smith and used for statistics and future model training.

The sending of reports to companies in the sector concludes the processing phase in the event that a threat (current or probable) is found. The report is then converted into a tweet and posted to my Twitter account.

The Server component was developed using ASP.NET Core and runs on Linux machines using a SQL Server 2019 database. We currently have three instances in three virtual machines managed by the same physical host. In the future, these services will need to be on physical hardware to improve performance.

The Agent component was developed using .NET Core and runs on Linux machines. Communication with servers takes place via HTTPS calls. We currently have nine instances each on a dedicated virtual machine. The machines are spread across two providers on four continents. The current load on these machines is 90% and to stay below this threshold it was necessary to limit some components, penalizing overall performance.

Currently the component writes its logs into files. In the future, these files will have to flow into a logs management platform to allow more immediate monitoring and the creation of alerts in the face of logged events.

Argilla

Argilla is my new project available on Github.

Argilla is a simple distributed bus for the integration between microservices. The adoption of Argilla allows the elimination of microservice endpoint configuration so that they can be managed more easily.

At present the project is functional even if it is not performing and unstable. Your help could be very useful to improve it 😉

The idea of this project starts from an internal need of my cyber security project developed in .NET Core on Linux. I needed to enable various services to communicate without having to manage a catalog and even worse a distributed configuration. I also had the need to have callbacks from calls that can last for hours.

Creating an Argilla enabled microservice is very simple, just add a reference, a few lines of code and your microservice are ready.

The architecture of Argilla is very simple, there is a Resolver server and its endpoint is configured in the various microservices. Argilla automatically publishes the catalog of services endpoint so that clients can consume the services without previously knowing their locations. If multiple services implement the same service, the client takes turns invoking the services to distribute the load. The client then takes care not to invoke the offline services and turn the requests to those available.

Wait for an event in a WaterfallDialog

Working on a Skill, I needed to block the execution of a WaterfallDialog while waiting for a specific event to arrive.
The solution is simple and based on the use of EventPrompt. Below I show how to use it.

Register EventPrompt in the constructor

AddDialog(new WaterfallDialog(nameof(WaterfallDialog), new WaterfallStep[]
{
	HelloAsync,
	WaitForEventAsync,
	DoneAsync
}));

AddDialog(new EventPrompt(nameof(EventPrompt), "SampleEvent", Validate));

InitialDialogId = nameof(WaterfallDialog);

Implement the three dialog steps

private async Task<DialogTurnResult> HelloAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
	await stepContext.Context.SendActivityAsync(MessageFactory.Text("Hello :)"), cancellationToken);

	return await stepContext.NextAsync(cancellationToken);
}

private async Task<DialogTurnResult> WaitForEventAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
	return await stepContext.PromptAsync(nameof(EventPrompt), new PromptOptions { Prompt = MessageFactory.Text("Waiting for the event") });
}

private async Task<DialogTurnResult> DoneAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
	await stepContext.Context.SendActivityAsync(MessageFactory.Text("After I receive the event."), cancellationToken);

	return await stepContext.NextAsync(cancellationToken);
}

Implement the event validator

async Task<bool> Validate(PromptValidatorContext<Activity> promptContext, CancellationToken cancellationToken)
{
	var activity = promptContext.Recognized.Value;
	if (activity.Type == ActivityTypes.Event)
	{
		try
		{
			string eventValue = (string)activity.Value;
			
			if(eventValue == "OK")
			{
				return true;
			}
		}
		catch { }
	}

	return false;
}

Share data from Virtual Assistant to Skill

For some weeks I have been working on a project based on Bot Framework. Things were going well until I found myself in the position of having to share information between a Virtual Assistant and a Skill … Given that it gave me problems and that my requests made on Stackoverflow did not produce answers, I share the solution here.

This post expects the user to be clear what Virtual Assistant and Skill are and has already installed the prerequisites as described here:
https://microsoft.github.io/botframework-solutions/tutorials/csharp/create-assistant/1_intro/

Create a Virtual Assistant using the Virtual Assistant Template and name it SampleVirtualAssistant

Add to the solution SampleVirtualAssistant a new project (the Skill) using the Skill Template and name it SampleSkill

Rebuild the solution.

In the following example I’ll show the commands as execute on my system, where my sources path is C:\Users\emili\source\lab, be aware to replace it with your correct path!

Open PowerShell Core to execute the following commands.

Deploy to Azure the required resources.

cd c:\users\emili\source\lab\SampleVirtualAssistant\SampleVirtualAssistant\SampleVirtualAssistant
.\Deployment\Scripts\deploy.ps1
cd C:\Users\emili\source\lab\SampleVirtualAssistant\SampleSkill\SampleSkill
.\Deployment\Scripts\deploy.ps1

Now we go to work on the code.

In the project SampleSkill, edit the file manifestTemplate.json to add the required slot.

cd C:\users\emili\source\lab\SampleVirtualAssistant\SampleVirtualAssistant\SampleVirtualAssistant
botskills connect --botName SampleVirtualAssistant --remoteManifest "http://<your resource>.azurewebsites.net/api/skill/manifest" --luisFolder C:\Users\emili\source\lab\SampleVirtualAssistant\SampleSkill\SampleSkill\Deployment\Resources\LU\en\ --cs

In the project SampleVirtualAssistant edit the file skills.json to add the required slot.

In the project SampleVirtualAssistant edit the file MainDialog.cs to update the class Events

In the same file, in the method OnEventAsync add the code that handle the event to set the UserEmail

In the project SampleSkill, in the file MainDialog.cs in the RouteAsync method insert this code after the call to PopulateSateFromSemanticAcion

Rebuild all.

Publish the skill and the VA.

Start the chat and test it 🙂

Azure Key Vault – Basic

Over time, I have used Key Vault several times. I think it’s a great solution and I clearly recommend its use to everyone!

Since every time I talk about it with something, after the enthusiasm I am asked how to use it, I have prepared a simple project that illustrates the basics of its use.

Use this PowerShell commands to create the Vault:

# Using this command you can see all the location available
az account list-locations

az login

# The following command a new resource group. If you already have one you can use it. 
# Choose your location
az group create -n "resource-group-name" -l "North Europe"

az provider register -n Microsoft.KeyVault

# Create the Key Vault. This call return the URL of the key vault.
az keyvault create --name "keyvault-name" --resource-group "resource-group-name" --location "North Europe"

# Add some secrets to the vault
az keyvault secret set --vault-name "keyvault-name" --name "secret-1" --value "test 1"
az keyvault secret set --vault-name "keyvault-name" --name "secret-2" --value "test 2"
az keyvault secret set --vault-name "keyvault-name" --name "secret-3" --value "test 3"

# List all the secrets in the specified vault
az keyvault secret list --vault-name "keyvault-name"

# Create an app. this call return the appId and secret to use in the app.config
az ad sp create-for-rbac -n "app-name" --skip-assignment

# Trust the key vault to be accessed with the app credentials
az keyvault set-policy --name "keyvault-name" --spn <use the appId previously created> --secret-permissions get list set

In this example, credentials are used to log in to the vault, I recommend, if you have an application in Azure, to set up an identity to log in to the vault.

Here you can find a .NET example project:
https://github.com/ecarlesi/azure-keyvault-sampleclient