Subsetting: not the whole database. Just part of it

Sometimes we don’t need to copy the entire production database onto a development or testing enviroment. It migth take a long time to do it and be really expensive in terms of processing and storage resources.

The solution is to select a subset of coherent data, mask it, and copy it onto the target environment. Maybe it is something like “10% of all customers, plus last month invoices”

icaria Mirage provides such capabilities. A subsetting path (or more than one) can be described: it means that the user must provide an starting point, probably a relevant entity like the customer, the invoice, the contract, the policy, or something like that, and then how to jump from one entity to another, so the subsetting path can be completed.

Therefore some knowlege of the data relationships must be known and reported to icaria Mirage.

Image. Setting up the subsetting path

Two out of our three projects requiere subsetting. As this capability was tested during the proof of concept activities, we have gathered all the avaliable information and deliverd it to the client team for rewiew.

Basically, the client team needs to think about the fucntional criteria to define the requiered data. Then they’ll have to decide what paths – relationship among data – are to be used to obtain a coherent set of information.

One of the client teams finds this last task challenging: they do not have enough knwledge nor documentation of the data model and application behaviour to describe the model relationships.

We migth offer some help. Maybe it’s time to test a new icaria Mirage plugin for automated data relationship discovery. It is not released yet, but if we give it a try, we risk nothing.


Sensitive Data Inventory

This is a key piece of information in the datamasking implementation project.

The purpose

We need this document so we know what information is the company interested in protecting.

Some of it migth be due to legal requirements, but there also migth be reasons for other business related information to be masked, regardless of its legal condition.

This document also gathers information about the appropriate datamasking algorithms. Some of them may be well-known in advance, and provided by icaria Mirage out-of-the-box. But some others are company or application specific.

Who is invited to participate

Generally speaking, at least the following opinions are required:

  • Information security responsible
  • Application functional leader
  • Software testing leader

The tools

We have provided the client team with a Sensitive Data Inventory template in Word format.

We are also considering to develop a new functionality within icaria Mirage to support this process


The ball starts to roll

After an exhaustive and sucessful Proof of Concept, the kick-off meeting is the first milestone of the real implementation project. The project team -client and netZima – had been working together for a while, so we know one another and the working atmosphere is great.

We are very confident this project will be a technical challenge (where is the fun, if otherwise?) but also a success. But first things, first. The kick-off meeting is not only about shaking hands and wishing the best for the task to come. Some important goals must be accomplished. This is what we did:

Introduce the team

Although many of us kew each other from the PoC, there were new faces. It is extremely important that everyone is properly introduce and everybody else knows what to expect from others, and how to reach to them. Share contact details.

Project highligths

Yes, everyone has readed the project SOW (statement-of-work), but it is best if we go through it together. So we did it. And we got useful information about potential issues and risks related with the planning, resource allocation, and people availability. So we had the opportunity to improve our project plan, and anticipate some issues.

We specifically shared the company goals set for this project. Clear and shared goals are paramount to project success.

We also review common project procedures and methodologies.

First assignments

There is plenty of work to do and not so much time. There are a few tasks that can be launched rigth away:

  • Fill in the Sensitive Data Inventory template
  • Review the data source perimeter
  • Have the hardware infrastructure available



Why risking your sensitive data for nothing?

In 2015, Jeb Bush published Reply All. A book that gathers a wide sample of email exchanges with Florida journalists, businessmen and everyday citizens while Bush was Florida governor. The book was meant to help relaunch a struggling campaign, by tapping on transparency.

But there was a problem.

As The Washington Post put it: “The e-mails all contain unredacted e-mail addresses, details about business operations and, according to the Verge, even Social Security numbers

This became an embarrassing issue for Jeb Bush’s attempt to became the Republican Party Candidate for the 2016 United States presidential election.

Was personal data relevant to the book purpose? Definitely not. It could have been redacted.

The same reasoning applies to personal data copied onto software development and testing databases.

What’s the need?

Software development and testing activities require rich, coherent data that looks real. They don’t need actual personal data.

But when you copy real data onto non-production environments security measures shrink while number of people with access to sensitive data, grows. It’s a lot more likely to suffer a data leakage that damages your company’s reputation.

Data masking technology provides cost-effective safe data for non-production environments.

Why risking it all?


icaria technology at Testing & Tools Day

icaria technology will be present at the V edition of Testing & Tools Day. We will be talking about GDPR and some of the solutions icaria Mirage provides when it comes to sensitive information data masking. We will share our most relevant and recent experiences working with some of the most important companies in Spain.

The event will take place at Madrid on October, 18.

We are looking to hear from you at icaria technology stand!


Three projects in one

icaria Mirage  has been recently selected as the data masking solution to protect sensitive data in non-productive environments by three major companies in the banking and insurance industries.

As these projects will be unfolding essentially at the same time, we thought it is a great opportunity to gather the lessons learned from the three of them in a journal. So this is what this blog is about. We won’t be talking about an specific project, but about all of them.

Our purpose is to offer useful insights of a data masking project based on icaria Mirage. Everyone interested in protecting sensitive data will find here information about what is it like. We believe that many of our experiences could apply also to other technology solutions.

Welcome. We are willing to start.

Picture by Oliver Tacke

No puedes usar datos de tus clientes para probar software sin su consentimiento

El pasado 24 de Mayo de 2016 se aprobó el nuevo  Reglamento General de Protección de Datos de la Unión Europea, con el objetivo de que exista una única normativa en todos los Estados miembros sobre la materia.

Las Entidades (empresas, Administraciones Públicas, ONG, o cualquier otro tipo de persona jurídica, incluidos profesionales) que dispongan de datos de ciudadanos europeos deberán cumplir con la nueva normativa a partir del 6 de Mayo de 2018, sin importar dónde se ubiquen.

Las sanciones por no cumplir son muy importantes: para infracciones muy graves pueden alcanzar los 20 millones de euros o el 4% de la facturación global.

Hay un aspecto de las nuevas obligaciones que puede tener mucho impacto en los procesos de pruebas de software de las empresas que utilizan datos reales de sus clientes: es necesario obtener consentimiento explícito del cliente para usar sus datos en pruebas de aplicaciones. Da igual que se disponga de un consentimiento genérico. No servirá a partir de mayo de 2018.

Y en el caso de que se obtenga el consentimiento explícito del cliente no hay que olvidar que a esos datos les aplica la Regulación General de Protección de Datos.

¿Y si recabar el consentimiento explícito para usar los datos de un cliente en pruebas de aplicaciones es demasiado costoso?

Luis Cisneros, asesor Jurídico TIC de Secure&IT y colaborador del blog especializado de la compañía, ofrece una solución viable: “La nueva regulación, en su considerando 78, establece que una medida para garantizar los derechos y libertades de las personas físicas puede ser la disociación”

La disociación de datos sensibles será la única posibilidad para usar datos reales en las pruebas de aplicaciones de forma legítima y sin necesidad de recabar el consentimiento del cliente Además a los datos correctamente disociados no les aplica la reglamentación.

icaria Mirage es la plataforma de identificación y disociación de datos sensibles, disponible para plataformas abiertas y Mainframe.