It’s all about YOW!

New Thinking 30 June 2014 Peter Sellars

Last December, I attended the YOW! 2013 Developer Conference in Brisbane, Australia. YOW! is a unique Conference that takes place over two days in three cities in a two-week period. It is the largest technology-agnostic developer conference in Australasia and invites leading experts and practitioners from across the globe, with industry-leading companies also well represented. High-calibre speakers are the norm and, in recent years, Don Reinertsen, Jean Tabaka, Jeff Patton, Jim Webber, Dan North, David Evans, Patrick Debois and Jez Humble have all spoken at the event.

A number of themes became apparent during the Conference. Taken in isolation, each theme provided individual benefits – but it is as a set of emerging practices that they will have the largest impact on solution delivery.

This set of emerging practices means that a consistent development environment can be streamlined and improved based on validated learning from fast production feedback. Micro services can benefit from the use of graceful failure handling and anomaly detection to make them viable alternatives to existing monolithic applications. Advances in anomaly detection can provide increased confidence in graceful fallback mechanism feedback and can provide deeper learning opportunities.

The foundation for these emerging practices is laid by the themes that came out of YOW! 2013, namely ‘Failure is inevitable. Accept it!’, ‘The Emergence of Improved Anomaly Detection’, ‘Recognition for Micro Services’, ‘Business Benefits associated with Validated Learning’ and ‘Dev is the New Production!’.

Failure is Inevitable. Accept it!

Accept that failure is inevitable. Failure in terms of hardware and software will occur. Switching the focus from prevention to graceful handling of, and recovery from failure is an emerging trend. Teams are also benefiting from the introduction of controlled failures to gain insights into existing system resilience.

Netflix deals with large-scale hardware and software failures by implementing techniques that prevent cascading failures and minimise impact to their consumers. Use of the circuit breaker and bulkheads stability patterns, tryable semaphores and functional reactive programming are among the techniques that contribute greatly to Netflix’s successful fault tolerance strategy. Chaos Monkey, a member of Netflix’s famous Simian Army tool suite, is being used by teams around the world to gain valuable insights into their own system failure tolerance.

The Emergence of Improved Anomaly Detection

Emergent anomaly detection tooling is providing additional value when analysing streams of data. Tools invoking learning algorithms are replacing previously static threshold and simple statistics-based implementations. Frequently changing thresholds no longer need to be administered as the new tools learn about expected system behaviours. Anomaly detection seems set to revolutionise the impact of real-time monitoring and alerting applications.

Grok is an example of this emerging toolset that utilises adaptive learning, automated modelling and complex pattern detection to gain insights into and knowledge of system behaviour. NuPIC (Numenta Platform for Intelligent Computing) is the open-source project extended by Grok and looks set to form the foundation of future tools.

Recognition for Micro Services

Micro Services are gaining recognition as a valid alternative to large monolithic applications. These services have a bounded context, favour choreography over orchestration, scale using processes not threads, don’t share code or use session state and typically consist of less than a hundred lines of code. Stability patterns such as timeouts, circuit breaker and bulkheads are utilised to prevent cascading service failures. Fred George led the Micro Services charge last year when he shared his uSwitch experience. This year, ThoughtWorks consultant Sam Newman dived deeper into the practical implementations and implications.

Business Benefits Associated to Validated Learning

Validated learning is gaining traction as a valuable feedback mechanism. Real-time data stream analysis combined with improved business intelligence tooling is enabling better-informed business decisions. Production usage patterns are now being used to provide input into future development.

A deeper understanding of customer behaviour and their interactions with relevant features is empowering companies to develop products that closer meet their customers’ needs. This inspection and adaptation concept popularised via the Lean Startup movement is providing valuable business insights for software development teams.

Dev is the New Production!

Development is production. Teams are reaping enormous benefits by using configuration management, provisioning and deployment tools to establish and maintain their development environments.

Developer environments are no longer works of art to maintain. They are disposable virtual machines or containers that are configured, provisioned and deployed in the same way as production. Tools such as Packer, Vagrant, Puppet and Fabric are being used by teams to provide these capabilities that enable faster feedback and diagnosis of configuration and environmental change impacts.

2014 is set to be another interesting year...

Interestingly, tools and methodologies related to these themes abound in the ThoughtWorks Technology Radar that is a respected indicator of emerging trends. From a personal perspective, it was pleasing to know some of the choices made in my Personal Technology Radar were in line with these emerging trends.