khalido.org

every blog needs subheader text

PyCon Australia 2019

Table of contents.

update this post to fix youtube video links

My notes for the PyConAU 2019 talks I went to. This blog is an attempt to try and capture some useful info from the talks.

Future challenge: Make a jupyter notebook for each talk to implement some of the learnings or tools or thingamajig learned.

The pyconAU19 videos are up on youtube.

You dont always need NumPy

The numerical Python ecosystem and communities are mature and powerful, but sometimes we can be too quick to reach for the numerical hammer when simpler options exist. This talk will outline some areas where the numerical stack may not be the best starting point, and survey some alternatives. #

with open('very_big_file.txt', 'r') as f:
  for line in f:
    results = (do_something(line) for line in f)

takeaway: document all the numerical things, and don't be afraid to mix and match. Whats the actual problem you're solving - does it matter if its taking 1ms in numpy vs 5ms in python?

Flying by the seat of our pants: our journey of teaching python using drones

A few years ago we found ourselves teaching students everything from Python to Arduino. It was our plan to replace all the other languages with Python as we streamlined our course. Learn why we introduced drones, what we learned not to do and what we were surprised about. #

takeaway: theres a lot to research and think about before even starting to code and fly a drone. I guess that applies to programing in general - so much problems in python are best approached after a literature research for best practices.

Highly visibile reactions are great, so try and use that with real projects as well.

Using Dash by Plotly for Interactive Visualisation of Crime Data

Dash is a great Python framework for building data visualisation websites. In this talk I discuss the framework basics, explore a sample site and describe its use for an enterprise application to graph crime statistics. I finish with clear pros and cons of the Dash framework from our perspective. #

takeaway: Just tried out plotly on my laptop, and it is a bit slow. Use it with caution as needed. It looks great for internal dashboard where you have beefy machines and fast connections to users, for personal use to dump on the internet a dash app is a hefty thing to host.

Building, designing, teaching and training simulation environments for Machine Learning

Imagine you’re building a fancy robot-driven warehouse. Your pick, place, and packing robots need to get around quickly, find the right item and put it to the right place without colliding with each other, shelves, or people. But you don’t have any robots yet, and you need to start. Try simulations! #

takeaway: Unity is easy to use, try it out with python!

Forecasting Australias 2019 Election with PyMC3

Can we predict the result of an Australian election before it occurs? How certain of the outcome can we be? My talk will use the 2019 Australian federal election as a case study to provide an entry-level introduction to the benefits of probabilistic forecasting and PyMC3. #

takeaway: forecasts with probabilities are much more useful than just a plain number. With the last Australian elections, practically every channel had just the one number for wins - but if they had attaced a likeihood to it and shown the other likely scenarios it would have been a lot more useful.

Apply probabilistic forecasting to a future issue. Read this book and ThinkBayes.

cuDF: RAPIDS GPU Accelerated Dataframe Library

RAPIDS open-source software enables end-to-end data science and analytics pipelines to run entirely on GPUs. Key to RAPIDS is cuDF, a pandas-like Python data frame library with a high-performance CUDA C++ implementation. cuDF and RAPIDS enable large speedups for end-to-end data science using GPUs. #

takeaway: should drastically speed up many of the bigger datasets I've tried in pandas, where a clean and transform pipeline would take many minutes. cuDF promises to at drastically speed up this process, making it easier to iterate faster.

So look into getting a Nvidia powered computer suitable for using with rapids, this would have drastically sped up some of my machine learning projects.

Understanding GPUs

With torch and tensorflow we have begun to rely on GPUs to speed up computations. Deep Learning or not, GPUs can provide massive computation speed ups but it’s not a panacea as NVIDIA would have you believe. Understanding how GPUs work can tell us where we should and shouldn’t use them. #

takeaways: very impressive talk. Worth rewatching if doing something with GPU's. For my next ML project consider getting my own gpu box running linux and look at libraries like rapids.

Not a long time ago, in a galaxy not very far away, an astronomer and a computer scientist walk into a bar...

Python is one of the most popular programming languages in astronomy. In this talk, I will tell a story about how Python helped me to develop a software tool for galaxy modelling, and tackle the scientific and technical challenges that arise in the Big Data era of astronomy. #

takeaway: Astronomers are using python for everything, and python can be fast with the right tools.

Learn to control your brain: Brain Computer Interfacing with Python

Neurofeedback is a brain-computer interface where a person’s own brain waves are audio/visually presented back in real-time after they’ve been recorded and filtered within a few milliseconds. We present methods to allow people to see their own brainwaves with python. #

takeaway: lots of interesting work being done. Interesting thing is that the neural data is pretty straightforward, no wonder teenagers are building brain controlled prosthetic arms these days!

Explore openbci further and hack my own neuro thingamajig.

Machine Learning and Cyber Security: Detecting malicious URLs in the haystack

Today, security teams are in an increasingly one-sided battle to defend against a myriad of cyber attacks. Web-based attacks are often devastating, with conventional blacklists and reputation-based defence tactics not able to identify previously unseen malicious URLs. Is AI the solution? #

takeaway: see the presentation again... it was a good explaination of a end to end process.

Creating Lasting Change

The Nature of Organisations, People, and how to Change Them #

takeaways: Its not just them, its us. Solve business problems and communicate how your solution helps. Don't spend too much time on we should have done it right in the first place. It just so happens that we adopt new tech as sometimes its better/faster/cheaper than the old.

Lessons learned building Python microservices

I will talk about challenges and wins that have come from introducing Python into a multilingual microservices kubernetes architecture with lots of legacy. #

- Richard Jones, dev at reecetech, a plumbing company

takeaway: make things simpler by taking away choices by using automated tools. Test and monitor services.

Tunnel Snakes Rule! Bringing the many worlds of Python together to monitor Melbournes biggest infrastructure project.

Python is being used to provide real-time environmental monitoring on the Melbourne Metro Tunnel project. Come along to see how open source Python tools from the web, IoT, cloud infrastructure and scientific domains are being used together to monitor environmental telemetry on a city-wide scale. #

takeaway: dang. that is a lot of stuff.

Using python programmed microcontrollers to build comedy props

Early-career comedians often have difficulties adding electronic props to their acts, due to the high cost of materials and fabrication skills required. This talk will recreate several props used in comedic performances, showing the code and components used. #

takeaway: explore micropython and get a device to play with.

Python Applications in Infrastructure Planning and Civil Engineering

Engineers tasked with planning new infrastructure constantly face the problem of having to look through too much information. This talk is about how we wanted to be lazy and wrote a bot to do it for us instead. #

takeaway: Impressive how a very simple NLP pipeline and application of XGBoost gave them such good usable results. They didn't even use word embeddings! There is a lesson in this. Build simple, improve later.

Fantastic Blocks And Where To Hide Them

Ruby has blocks. JavaScript has blocks. Swift has blocks. Python doesn’t have blocks.In this talk, we’ll look at why Python doesn’t have blocks, and recent programming techniques that have developed in languages that do have blocks. Then we’ll look at what we – or Python – can do about it! #

takeaway: things to thing about. Convention matters.

Profiling Pathogens with (micro) Python

We’re building professional medical diagnostics equipment with micropython. This has come with minimal challenges, many positives and a few surprises! #

takeaway: use the jupyter kernel for micropython for live dev on a board.

The Antipodes

takeaway: watch more talks by great talkers. Put the important stuff first, both in programming and emails.

Saturday Lightning Talks

Sam Bishop: Computational World Building

data driven look at code comments

logging

The real costs of Open Source Sustainability

What if money isn’t the only way to create sustainable free and open source software projects? What if it turns out that sustainability is actually a multi-faceted concept that can’t truly be successful if people focus on only one of its many elements? #

takeaway: code of conducts matter, look for, abide by and try to contribute to open source projects which have them.

Shipping your first Python package and automating future publishing

One of the best things about Python is the vast ecosystem of packages available on the Python Package Index. Shipping your first Python package can be intimidating. This talk aims to remove the mystery of Python packaging and enable you to share your code with the Python community. #

takeaway: publish my first package. use nox and circleci to automate all the things.

Insights into Social Media Data using Entropy Theory

Entropy theory is usually thought of as something that applies to matter and energy, but it turns out that we can apply the same techniques of analysis to social media sites. Join me as we study the thermodynamic behaviour of users on Twitter, and learn how to analyse it better. #

takeaway: Look at other fields for ideas on how to tackle problems.

It's dark and my lights arent working (an asyncio success story)

I have invested huge amounts of time in achieving a simple goal – making the lighting in my home “smart”. It’s not ground breaking, nor is it practical or cost effective, but it sure was educational, uses a bunch of Python, and the result makes me (and my family) happy. #

takeaway: home automation is a deep deep rabbit hole to jump into. This guy should get a Medal for Bravery above and beyond the call of duty from Zigbee and Samsung smartthings.

The universe as balls and springs: molecular dynamics in Python

Surprisingly, we can approximate matter as a bunch of balls on springs and learn things about our bodies and the world. This talk will look at the different stages of molecular dynamics (MD) simulations and how Python is changing everything. #

takeaway: molecular dynamics is hard. I liked how they simplified a very complex thing - proteins, viruses - to a simple system which they could model computationaly and hope it approximates the real thing.

Instant feedback, instant-debugging Python coding

Building on Bret Victor’s famous ‘Inventing on Principle’ presentation, we look at writing Python where the code is instantly run and every line visualized after every single keystroke. There’s a future beyond the text-editor -> console-run loop and this is a taste of it. #

takeaway: instant feedback is genius. I want a instant feedback python IDE. Actually strip the ide. Just a simple two pane interface, one showing code, other the results. Someone pls build a vscode or jupyter extension for this. I mean ppl are sending keystrokes to cloud AI's to predict smart suggestions, just running the code must be easier than that!

Orchestrating complex (not complicated) tasks using AWS serverless and Python

Python and serverless technologies are a great way to quickly scale a project, but what can you do when things get complicated? Here are some patterns to keep you sane. #

takeaway: instead of making overly complex lambda functions, model the problem as a finite state machine, make simple lambdas as appropriate then make a state machine which embeds some of the logic and calls lambdas.

Then you can make a state machine to watch over all your other state machines!

Sunday Lightning Talks

Pre european potatoes

Quantum Computing

Batteries included but leaking

mega takeaway

Its nice to hear talks in real life instead of on youtube. Focuses the attention.

Moving forward, my biggest takeaway is to take an idea and just run with it. Many of the speakers had an itch - a home to automate with python, a paper to write or drones to fly and just did it. So take one of my project ideas and just do it, using circleci of course.

misc notes

posted , updated
tagged: python View on: github