I've been privileged to work with talented engineering teams for over forty years. They have varied from startups to large international companies. In this talk, I discuss key lessons which I learned from great and terrible technical leaders.

We look at the many hats of a CTO - Engineering Lead, Engineering Director, Product Owner, Technical Visionary, R&D Director, Chief Entrepreneur, Playing Coach, Chief Scientist, Distinguished Engineer, Board Member, Community Leader. We discuss the essential qualities of technical leadership. Why the Vision thing is an essential feature of a good CTO. The importance of your global technical network.

Despite order of magnitude advances in hardware, significant improvements in software development and execution remain elusive. Modern practices of Continuous Delivery; Functional Programming; Microservices; Data Driven; NoSQL; Event Sourcing; Accelerated Development; Mobility; IoT; SAAS are only possible due to significant improvements in hardware.

What if FAST Hardware => FAST Software? Is this talk we look at ways in which simple design allows us to exploit the power of modern hardware. We begin with a quick look at hardware today and tomorrow and why software slows it down! We the argue through examples that aligning the software and hardware architecture often leads to much simpler, and faster solutions.

Technology gurus predict that in 2017 – 2018 time frame we have computers with 10 terabytes of non-volatile memory! In this talk we explore the disruptive impact of massive amounts of memory and try to answer the following questions. What are the potential benefits and costs of all this free and persistent memory? Will be have database or file systems or transaction read/write sets or Java style garbage collection? What happens to application architectures and development in the huge memory world? How can large memory make development better and execution faster?

For decades, many have argued for and against Dearden’s “Myth of real-time management information” HBR 1966. At best most organizations have had only partial data, and insufficient storage and processing, limiting their ability to be a real-time enterprise. Thanks to massive compute power, memory and storage it is now finally possible for modern enterprises to capture and process the totality of their data current and past. Given they have real-time data the challenge becomes how to clean, transform, query, analyze and present this in meaningful, actionable real-time information. In this talk, we discuss the challenges of working with the real-time data deluge from sensor IoT to Big Data.

We take a brief look at the world of big data tools, event processing, and skills as they exist today. We then discuss our experience building Ivy, an interactive visual analytics environment for Thinkers. Thinkers are the key enablers for a real-time enterprise and need a capability to Think-Compute-See and actively participate in problem solutions. We discuss user requirements for acquisition, preparation and visualization and our challenges developing solutions consumable by Thinkers that are built on state of the art software technologies fast data technologies.

IoT devices vary from extremely limited 8/16/32 processors to Linux capable machines. In order to quickly develop products/applications, one needs higher level tool chain for productivity. In this talk, we describe the essential features and constraints of programming models for different device classes. Device programming typically requires substantial low-level coding which often needs to be substantially rewritten for different processor configurations. Productivity and portability are often achieved by using discipline programming techniques and high level language abstractions. We then discuss our experience with table driven interpreters, procedural, object and vector virtual machines and associated tooling.

For the last decade, there has been increased concern about the integrity of capital markets. The crash of 2008-2009 and follow legal actions and press have created an image of a world of high-frequency traders who can leverage their computer power to manipulate markets. Technical talks on performance which is critical in finance, further characterize finance as hooked on speed/low latency. One gets the impression that fast data leads to a fast buck at public expenses. However, fast big data also enables the good guys!

We discuss how fast big data is being used in the financial industry to ensure good governance and protect consumers and businesses who depend on the integrity of financial markets. We discuss the better decisions enabled by algorithms; improved testing practices for algorithms; oversight of markets through surveillance; protection against cyber threats; and the use of data forensics to tell the true story of transactions past.

This talk explores big data using vector functional programming. These languages enable rapid interactive development of concise and efficient applications. Vectors are a natural low impedance abstraction for dealing with high performance relational and NoSQL column stores. We use the k language and kdb+ database to illustrate the vector applicative style and idioms of FP through examples working efficiently with big data.

Vector languages are heavily used in finance and insurance for analysis of large time series. Vector/Array dynamic languages originated with APL family and include NIAL, J and k and databases such as Kdb+, MonetDB and Vectorwise. The have inspired array extensions in other general languages, such as Python NumPy and NamedTuple; and special purpose GPUs. Often these languages are considered high barrier languages because the compact notation and vector thinking require an investment to develop proficiently. Broaden your FP knowledge and understand the joy and power vector FP.

Dave Thomas: NewSQL - The New Universal Query Language for Everything.

Back to the Future! In 2020 many new and legacy data systems, such as Spark, Teradata, Mongo, Cassandra, Splunk, Kafka, Flink, Azure, Google, Oracle…, will support a NewSQL.

We begin with a very brief review of the current state of data engineering practice - NoSQL, Big Data, Streaming, Time Series, BASE versus ACID, Graphs; DSLs, Map Reduce, Functional Programming etc. We look at the challenges presented by the plethora of different data formats, languages/tools and eventual consistency.

What if we didn’t have to deal with the challenges of sharding and eventual consistency? We discuss the recent increased use of new distributed ACID databases such as Google Spanner, AWS AuroraDB, Azure CosmosDB etc.

What if we didn’t need to worry about different languages, APIs for programming different databases? What if we could use one language for both batch and streaming? Very recently, at the ACM SIGMOD 2019 database conference, a new emerging data language was presented which seeks to provide a solution.

We refer to this emerging standard as NewSQL, which extends SQL to deal with both streaming and batch. It removes major limitations in SQL and provides the additional capabilities to SQL needed to deal with the velocity, volume and variety of diverse data sources. In this talk, we describe the evolution and major features of the new language. We conclude with a brief discussion of the impact on data engineering, data science and data consumers.

Danial Tham: Mystique - The Fight Against Rego Plate Cloning

In the rise of AI technologies, there is no doubt that privacy is a very popular topic which normally revolves around the negative impact of AI to our privacy. However despite the norm, we at Carsales built and deployed an AI tech called Mystique which fights crimes and protects user privacy. Mystique detects a rego plate in a photo and blurs it, protecting our private seller and dealer from rego plate cloning.

Rego plate cloning “involves falsifying a genuine number plate and attaching it to another vehicle which is often the same make, model and colour. The criminal may then use the car with cloned plates to drive on tollways without consequences, drive dangerously or conduct other criminal activity, with the genuine vehicle owner often sought to pay fines or tolls.”

In this talk, we are going to share the story about how we built and delivered Mystique and more importantly what lessons we learned in deploying tech at this scale, which processes 150,000 photos a day and touches many areas of business at Carsales.

Dave Thomas: NewSQL - The New Universal Query Language for Everything.

Back to the Future! In 2020 many new and legacy data systems, such as Spark, Teradata, Mongo, Cassandra, Splunk, Kafka, Flink, Azure, Google, Oracle…, will support a NewSQL.

We begin with a very brief review of the current state of data engineering practice - NoSQL, Big Data, Streaming, Time Series, BASE versus ACID, Graphs; DSLs, Map Reduce, Functional Programming etc. We look at the challenges presented by the plethora of different data formats, languages/tools and eventual consistency.

What if we didn’t have to deal with the challenges of sharding and eventual consistency? We discuss the recent increased use of new distributed ACID databases such as Google Spanner, AWS AuroraDB, Azure CosmosDB etc.

What if we didn’t need to worry about different languages, APIs for programming different databases? What if we could use one language for both batch and streaming? Very recently, at the ACM SIGMOD 2019 database conference, a new emerging data language was presented which seeks to provide a solution.

We refer to this emerging standard as NewSQL, which extends SQL to deal with both streaming and batch. It removes major limitations in SQL and provides the additional capabilities to SQL needed to deal with the velocity, volume and variety of diverse data sources. In this talk, we describe the evolution and major features of the new language. We conclude with a brief discussion of the impact on data engineering, data science and data consumers.

Agustinus Nalwan: Mystique - The Fight Against Rego Plate Cloning

In the rise of AI technologies, there is no doubt that privacy is a very popular topic which normally revolves around the negative impact of AI to our privacy. However despite the norm, we at Carsales built and deployed an AI tech called Mystique which fights crimes and protects user privacy. Mystique detects a rego plate in a photo and blurs it, protecting our private seller and dealer from rego plate cloning.

Rego plate cloning “involves falsifying a genuine number plate and attaching it to another vehicle which is often the same make, model and colour. The criminal may then use the car with cloned plates to drive on tollways without consequences, drive dangerously or conduct other criminal activity, with the genuine vehicle owner often sought to pay fines or tolls.”

In this talk, we are going to share the story about how we built and delivered Mystique and more importantly what lessons we learned in deploying tech at this scale, which processes 150,000 photos a day and touches many areas of business at Carsales.

Dave Thomas: NewSQL - The New Universal Query Language for Everything.

Back to the Future! In 2020 many new and legacy data systems, such as Spark, Teradata, Mongo, Cassandra, Splunk, Kafka, Flink, Azure, Google, Oracle…, will support a NewSQL.

We begin with a very brief review of the current state of data engineering practice - NoSQL, Big Data, Streaming, Time Series, BASE versus ACID, Graphs; DSLs, Map Reduce, Functional Programming etc. We look at the challenges presented by the plethora of different data formats, languages/tools and eventual consistency.

What if we didn’t have to deal with the challenges of sharding and eventual consistency? We discuss the recent increased use of new distributed ACID databases such as Google Spanner, AWS AuroraDB, Azure CosmosDB etc.

What if we didn’t need to worry about different languages, APIs for programming different databases? What if we could use one language for both batch and streaming? Very recently, at the ACM SIGMOD 2019 database conference, a new emerging data language was presented which seeks to provide a solution.

We refer to this emerging standard as NewSQL, which extends SQL to deal with both streaming and batch. It removes major limitations in SQL and provides the additional capabilities to SQL needed to deal with the velocity, volume and variety of diverse data sources. In this talk, we describe the evolution and major features of the new language. We conclude with a brief discussion of the impact on data engineering, data science and data consumers.

Agustinus Nalwan: Mystique - The Fight Against Rego Plate Cloning

In the rise of AI technologies, there is no doubt that privacy is a very popular topic which normally revolves around the negative impact of AI to our privacy. However despite the norm, we at Carsales built and deployed an AI tech called Mystique which fights crimes and protects user privacy. Mystique detects a rego plate in a photo and blurs it, protecting our private seller and dealer from rego plate cloning.

Rego plate cloning “involves falsifying a genuine number plate and attaching it to another vehicle which is often the same make, model and colour. The criminal may then use the car with cloned plates to drive on tollways without consequences, drive dangerously or conduct other criminal activity, with the genuine vehicle owner often sought to pay fines or tolls.”

In this talk, we are going to share the story about how we built and delivered Mystique and more importantly what lessons we learned in deploying tech at this scale, which processes 150,000 photos a day and touches many areas of business at Carsales.

NoSQL is slaying the Relational Database (RDB) dragons with their fast columns, simple key values, graphs, BASE, and schema less features throwing their hooked on ACID SQL row farmers and DBAs into disarray. But Wait! Rumours are emerging of a secret YesSQL tech heading at warp speed to the lucrative Copious Data Galaxy. Is NewSQL just RDB vendor FUD or does the Empire have any hot cool stuff? Attendees may choose be seated in columns or rows according to their preference!

In this talk we look at the exciting evolution of databases taking place to deal the challenge of copious data. Attendees will learn about the innovations of new database technology and the opportunities and challenges for using it in production and analytics applications.

NoSQL is slaying the Relational Database (RDB) dragons with their fast columns, simple key values, graphs, BASE, and schema less features throwing their hooked on ACID SQL row farmers and DBAs into disarray. But Wait! Rumours are emerging of a secret YesSQL tech heading at warp speed to the lucrative Copious Data Galaxy. Is NewSQL just RDB vendor FUD or does the Empire have any hot cool stuff? Attendees may choose be seated in columns or rows according to their preference!

In this talk we look at the exciting evolution of databases taking place to deal the challenge of copious data. Attendees will learn about the innovations of new database technology and the opportunities and challenges for using it in production and analytics applications.

NoSQL is slaying the Relational Database (RDB) dragons with their fast columns, simple key values, graphs, BASE, and schema less features throwing their hooked on ACID SQL row farmers and DBAs into disarray. But Wait! Rumours are emerging of a secret YesSQL tech heading at warp speed to the lucrative Copious Data Galaxy. Is NewSQL just RDB vendor FUD or does the Empire have any hot cool stuff? Attendees may choose be seated in columns or rows according to their preference!

In this talk we look at the exciting evolution of databases taking place to deal the challenge of copious data. Attendees will learn about the innovations of new database technology and the opportunities and challenges for using it in production and analytics applications.

It appears that there is a complexity conspiracy seeking to make the development of even straightforward applications difficult.

To be successful today one needs to cope with the full-stack - a constantly changing collection of languages, APIs, tools and frameworks.

It appears that complexity begets complexity? Yet most applications are not intrinsically complicated!

In this talk, we look at simple proven techniques which provide improved productivity and quality. They have stood the test of time, but unfortunately, are not well known to most developers. These techniques reduce the gap between the product owner and developer usually reducing amount code which must be developed. They have been applied to problems in almost every industry banking to real-time control. They don't depend on a particular language or technology hence large portions of an application can be moved easily from legacy technology to modern technology. We discuss the techniques and provide case studies of their use in major applications.

It appears that there is a complexity conspiracy seeking to make the development of even straightforward applications difficult.

To be successful today one needs to cope with the full-stack - a constantly changing collection of languages, APIs, tools and frameworks.

It appears that complexity begets complexity? Yet most applications are not intrinsically complicated!

In this talk, we look at simple proven techniques which provide improved productivity and quality. They have stood the test of time, but unfortunately, are not well known to most developers. These techniques reduce the gap between the product owner and developer usually reducing amount code which must be developed. They have been applied to problems in almost every industry banking to real-time control. They don't depend on a particular language or technology hence large portions of an application can be moved easily from legacy technology to modern technology. We discuss the techniques and provide case studies of their use in major applications.

Coming Soon...

And Now For Something Completely Different?

What do stock ticker feeds, web click streams, micropayments, online games, smart meters, and sensors on a formula one race car have in common? They all need to compute over and visualize large volumes of time series data. In this talk how one can interactively explore and visualize massive data sets using a vector functional programming.

Vector languages are heavily used in finance for the analysis of stock market data. They have inspired array capabilities in other languages, such as R, MatLab, Python NumPy, and Pandas. Vector languages and runtimes enable rapid interactive development of concise and efficient applications. Vectors are a natural low impedance abstraction for dealing with high-performance column stores. We introduce the key concepts of vector programming, applicative style, and idioms of FP. We then illustrate their use on typical data science problems exploring large data sets using visual analytics.

And Now For Something Completely Different?

What do stock ticker feeds, web click streams, micropayments, online games, smart meters, and sensors on a formula one race car have in common? They all need to compute over and visualize large volumes of time series data. In this talk how one can interactively explore and visualize massive data sets using a vector functional programming.

Vector languages are heavily used in finance for the analysis of stock market data. They have inspired array capabilities in other languages, such as R, MatLab, Python NumPy, and Pandas. Vector languages and runtimes enable rapid interactive development of concise and efficient applications. Vectors are a natural low impedance abstraction for dealing with high-performance column stores. We introduce the key concepts of vector programming, applicative style, and idioms of FP. We then illustrate their use on typical data science problems exploring large data sets using visual analytics.

And Now For Something Completely Different?

What do stock ticker feeds, web click streams, micropayments, online games, smart meters, and sensors on a formula one race car have in common? They all need to compute over and visualize large volumes of time series data. In this talk how one can interactively explore and visualize massive data sets using a vector functional programming.

Vector languages are heavily used in finance for the analysis of stock market data. They have inspired array capabilities in other languages, such as R, MatLab, Python NumPy, and Pandas. Vector languages and runtimes enable rapid interactive development of concise and efficient applications. Vectors are a natural low impedance abstraction for dealing with high-performance column stores. We introduce the key concepts of vector programming, applicative style, and idioms of FP. We then illustrate their use on typical data science problems exploring large data sets using visual analytics.

For the last decade, there has been increased concern about the integrity of capital markets. The crash of 2008-2009 and follow legal actions and press have created an image of a world of high-frequency traders who can leverage their computer power to manipulate markets. Technical talks on performance which is critical in finance, further characterize finance as hooked on speed/low latency. One gets the impression that fast data leads to a fast buck at public expenses. However, fast big data also enables the good guys!

We discuss how fast big data is being used in the financial industry to ensure good governance and protect consumers and businesses who depend on the integrity of financial markets. We discuss the better decisions enabled by algorithms; improved testing practices for algorithms; oversight of markets through surveillance; protection against cyber threats; and the use of data forensics to tell the true story of transactions past.