Understanding AI in a world of big data

Richard Boire

doi:10.3934/bdia.2018001

Article Contents

2018, Volume 0. Doi: 10.3934/bdia.2018001

Previous Article Next Article

Early Access

Early Access articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Early Access publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Early Access articles via the “Early Access” tab for the selected journal.

Understanding AI in a world of big data

Richard Boire

Environics Analytics, 33 Bloor St. East, Toronto, Ont. M4W3H1, Canada

Early access: October 2018

Abstract Full Text(HTML) Figure(19) Related Papers Cited by

Abstract

Big Data and AI are now very popular concepts within the public lexicon. Yet, much confusion exists as to what these concepts actually mean and more importantly why they are significant forces within the world today. New tools and technologies now allow better access as well as facilitating the analysis of this data for better decision-making. But the discipline of data science with its four-step process in conducting any analysis is the key towards success in both non-advanced and advanced analytics which would, of course, include the use of AI. This paper attempts to demystify these concepts from a data science perspective. In attempting to understand Big Data and AI, we look at the history of data science and how these more recent concepts have helped to optimize solutions within this 4 step process.

Keywords:

Mathematics Subject Classification: 91C20, 62F07.

Citation:

FullText(HTML)

Figure 1. [1] The 5 V's of Big Data

Download: Full-size image PowerPoint slide

Figure 2. [2] Moore's Law

Download: Full-size image PowerPoint slide

Figure 3. [3] Columnar File Format

Download: Full-size image PowerPoint slide

Figure 4. Example of Structured Data

Download: Full-size image PowerPoint slide

Figure 5. Example of Twitter Data

Download: Full-size image PowerPoint slide

Figure 6. Sequential vs. Parallel Data Processing

Download: Full-size image PowerPoint slide

Figure 7. Schematic of Simple Neural Net-One Hidden layer

Download: Full-size image PowerPoint slide

Figure 8. [7] Schematic of Weights within Neural Net Structure

Download: Full-size image PowerPoint slide

Figure 9. [8] Examples of some Optimization Algorithms

Download: Full-size image PowerPoint slide

Figure 10. Examples of Neural Nets

Download: Full-size image PowerPoint slide

Figure 11. Sample of 3 records

Download: Full-size image PowerPoint slide

Figure 12. Sample of 3 records-Fixed

Download: Full-size image PowerPoint slide

Figure 13. Frequency Distribution of Numeric Variable

Download: Full-size image PowerPoint slide

Figure 14. Frequency Distribution of Character Variable

Download: Full-size image PowerPoint slide

Figure 15. Example of Data Diagnostics

Download: Full-size image PowerPoint slide

Figure 16. Example of Alteryx Software

Download: Full-size image PowerPoint slide

Figure 17. Example of Gains/Decile Table

Download: Full-size image PowerPoint slide

Figure 18. Example of Final Model Variable Contribution Report

Download: Full-size image PowerPoint slide

Figure 19. Example of Final Model Variable Contribution Report

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	Figure.1: The 5 V's of big data, Environics Analytics: Best Practices and Considerations in Big Data Analytics, June, 2018.
[2]	Figure.2: Moore's Law, https://www.google.ca/search?hl=en&tbm=isch&source=hp&biw=1366&bih=651&ei=wd3pWuPdMqqPjwSUr4SQDQ&q=exponential+growth+in+computing+power&oq=growth+in+computing+power&gs_l=img.1.1.0j0i5i30k1.4574.14021.0.16389.28.27.0.1.0.0.182.2657.16j10.26.0....0...1ac.1.64.img..1.25.2467.0..0i24k1j0i8i30k1.0.eDlGB4j2AdI#imgrc=jhm-BdlhnmB2HM:.
[3]	Figure.3: Columnar file formats, https://www.google.ca/search?hl=en&tbm=isch&source=hp&biw=1366&bih=651&ei=wd3pWuPdMqqPjwSUr4SQDQ&q=exponential+growth+in+computing+power&oq=growth+in+computing+power&gs_l=img.1.1.0j0i5i30k1.4574.14021.0.16389.28.27.0.1.0.0.182.2657.16j10.26.0....0...1ac.1.64.img..1.25.2467.0..0i24k1j0i8i30k1.0.eDlGB4j2AdI#imgrc=jhm-BdlhnmB2HM:.
[4]	Index compression, https://nlp.stanford.edu/IR-book/html/htmledition/index-compression-1.html.
[5]	Figure.6-Sequential vs. parallel data processing, https://www.google.ca/search?biw=1607&bih=678&tbm=isch&sa=1&ei=UVPwWu_uGoeYjwSkqrjwBA&q=sequential+db+processing&oq=sequential+db+processing&gs_l=img.3...0.0.0.123836.0.0.0.0.0.0.0.0..0.0....0...1c..64.img..0.0.0....0.jkNEKg1fCW0#imgdii=kH8ag2orN-LWNM:&imgrc=pBOBcUMsqlXNGM:&spf=1525699534175.
[6]	Turn to in-memory processing when performance matters, https://searchdatacenter.techtarget.com/feature/Turn-to-in-memory-processing-when-performance-matters.
[7]	Figure.8: Schematic of weights within neural net structure, https://www.google.ca/search?hl=en&tbm=isch&source=hp&biw=1366&bih=651&ei=bpvwWv2FM82O5wLqzLigCA&q=neural+net+simple+network&oq=neural+net+simple+network&gs_l=img.3...1065.21853.0.22452.38.24.0.14.14.0.120.1836.21j2.23.0....0...1ac.1.64.img..1.13.1052.0..0j0i24k1j0i10i24k1j0i10k1j0i7i30k1.0.nu7gREvNHkk#imgrc=13gO7BFb0GYZqM:.
[8]	Figure. 9-Examples of some optimization algorithms, https://www.google.ca/search?hl=en&tbm=isch&q=logistic+function&chips=q:logistic+function,g_5:logistical&sa=X&ved=0ahUKEwjw-KD5oPTaAhWkpFkKHSxSDJwQ4lYIMCgA&biw=1366&bih=651&dpr=1#imgrc=oAHIGiD5uTjw2M: https://www.google.ca/search?hl=en&tbm=isch&q=tan+function+graph&chips=q:tan+function+graph,g_1:tangent,online_chips:cos+tan&sa=X&ved=0ahUKEwjK-IGYovTaAhVQwlkKHUBnC0cQ4lYIKygC&biw=1366&bih=651&dpr=1#imgrc=gWnErav-9CIbGM:.
[9]	"Is predictive analytics for marketers really that accurate?", Journal of Marketing Analytics, May, 2013. https://link.springer.com/article/10.1057/jma.2013.8.
[10]	"Data Mining for Managers: How to use data (big and small) to solve business problems", by Palgrave Macmillan, Oct, 2014.