Understanding Data Types for Effective AI Solutions

Categorizing data for AI use

We have already touched on the challenges that any engineer, and thus any engineering company, faces, which might be solved by a smart application of AI technologies. You can learn more about them from the following article: Use cases for optimizing engineering workflows with AI

As we have moved past the first step of dealing with a challenge—recognizing that it exists—we are poised for the next phase: trying to find a solution, examining what might be applicable from both a technological and an economic perspective.

To reach this point, I would begin by analyzing the data sources, as this forms the basis for any further automation and optimization of our processes.

Our data sources can be grouped, for all practical purposes, into following groups:

Structured data (SQL databases and similar, spreadsheets, …)
Unstructured data (Text documents, Images, Videa, Audio, Social Madia, …)
Semi-structured data (NoSQL databases, web pages, JSON, XML, email, …)
Time series data (Anything that comes with a time stamp: sensor readings, server logs, market prices, other statistical data)

Why are data categories according to structure important

According to the nature of the given data structure, we can estimate the approach and technology to apply; for example:

A deterministic approach to automation is a rule-based system where every action is precisely defined by a set of fixed, logical instructions (e.g., “if X happens, then execute Y”). Given the exact same input, it will always produce the same output without any variation or uncertainty. It does not learn or adapt; it strictly follows pre-programmed rules, making it ideal for tasks that are repetitive, well-understood, and require absolute consistency, such as data validation, workflow routing, or calculation execution.

Then better the data is structured, more efficient can be the use of AI resources from the point of token usage. We can also rely more on deterministic methods, as the steps of data extraction and classification are already performed.

A probabilistic approach to automation is a data-driven system that uses statistical models and machine learning to handle uncertainty, patterns, and predictions. Instead of following fixed rules, it estimates the likelihood of possible outcomes based on training data (e.g., “there is an 85% probability that this document is a URS”). Given the same input, it may produce different results over time as it learns from new data. It is ideal for ambiguous, complex, or variable tasks like natural language understanding, image recognition, anomaly detection, and forecasting, where rigid rules are insufficient.

On the other side, semi structured and unstructured data are obvious use cases for probabilistic methods, e.g. AI. Only Probabilistic methods can analyse and make sense out of scattered and unorganized data as it can be found in email threads, various human-made documents, designs, and similar

Let’s structure our example

In the above-mentioned article, we already touched upon eligible sources of data for our automation purposes. Let’s list them here and categorize them properly:

📊 Structured Data

(Organized in rows and columns with a fixed schema)

Excel tables (packaged with exported project data)
ERP data (fetched with SQL queries)
Access and similar “desktop“ databases (yes, it is still a thing)
On-line databases (Like MS Dataverse, Podio, Buildbase, NocoBase,…)

📄 Unstructured Data

(No predefined format; primarily free text, images, or binary files)

Layouts (visual design files/drawings)
Client correspondence (emails, project management notes)
Internal documentation (standards, process descriptions, instructions)
Notes (digital or digitized/scanned text)
Designs (e.g., CAD files)
Email inbox & chat history

🔀 Semi-structured Data

(Has some organizational properties but is flexible)

Product specifications (internal – Word/PDF with mixed content)
Quotes (internal – documents with flexible structure)
User Requirement Specifications (URS) (client-generated – bloated with legal text, varying wording, and embedded elements, including the ones in Excel format)
Other documents (general project files with mixed formatting)

📈 Time Series Data

(Temporally ordered measurements – not explicitly mentioned in the article)

Time related measurements and software logs – not mentioned in the article cited above, but thinking about it… by doing industrial automation, SCADA systems and controller software produce all kind of logs which can be collected and analyzed for debugging, maintenance, and development purposes

And now what?

Now that we know what information is at our disposal and we have classified it properly, we are ready for the steps that will follow. That is business case analysis and later the selection of applicable technologies for our case.

iGeneering