Not Small - Not Big Data: the Missing Size in the Data Spectrum

Abstract

This paper addresses the classification of data sets that do not fit into the traditional categories of Small Data or Big Data, introducing an intermediate category called Not-Small-Not-Big Data (NoS-NoB Data). Understanding and effectively handling NoS-NoB Data is crucial for various fields of data science, as it encompasses structured, semi-structured, and unstructured information that challenges conventional computing environments due to hardware and software limitations. NoS-NoB Data is primarily defined by its volume, which surpasses the capacity of standard office tools yet does not necessitate the complexity of distributed big data technologies. Typically ranging from tens of gigabytes to several terabytes, such data require specialized processing approaches, including advanced database management systems and programming languages with dedicated data analysis libraries. Processing NoS-NoB Data effectively demands a combination of relational databases, NoSQL solutions, and computational tools such as Python, R, and, in some cases, Hadoop or Spark, depending on specific analytical requirements. The application domains of NoS-NoB Data span various industries, including small and medium enterprises (SMEs), healthcare, education, retail, financial services, logistics, and public administration. This study summarises the fundamental distinctions between data categories based on volume, structure, processing methods, and technological requirements. Additionally, a visual representation illustrates the relationships between Small Data, NoS- NoB Data, and Big Data, highlighting the overlap between categories and the proportional representation of structured, semi-structured, and unstructured data.