Do it wrong so you can do it right

Photo by Glen Carrie on Unsplash

Through the years I have come across quite a few import issues and only got them to work based on trial and error. Imports are difficult because there are so many factors interacting with each other to make things work when they shouldn’t, and make things fail/get a warning when they should not. I’m finally tired of that and this article (mainly the loom video) aims to show you the factors affecting the success and failure of imports through a series of experiments that demonstrate many situations you may come across.

It may not cover all cases, but lays out…

Just don’t use the same alias name as input column name

Photo by Randy Fath on Unsplash

I came across code in SQLite (this article applies to Postgres too)

CASE WHEN name < 'c' THEN 'combined'
ElSE name
END as name

where the column alias (name after END AS) of a CASE statement used the same alias name as the raw input column name (name after WHEN), leading to possibly confusing results for beginners. This article explains what is happening in such cases with a short experiment at!7/2676c/3.

If trying your own tables, write the DML SQL and click “Build Schema” first before running queries with “Run SQL”.

Starting Table

| name | value |
| a…

Be careful with argument order and return values

Photo by Andrea Ferrario on Unsplash

I was working on this leetcode question using backtracking and spent some time debugging strange output. This article discusses some pitfalls when using recursion with global variables, how to handle them, and how to change the code from global to local.

Disclaimer: Backtracking only passing 15/75 test cases and Time Limit Exceeded for the rest, the purpose of this article is to highlight possible issues with global variables rather than give the best solutions. For more beautiful solutions using Binary Search, Dijkstra and Kruskal, watch Alex’s walkthrough (beginning 7:26) at


Hit the nail on the head

cmdCopy as cURL(bash) to formulate a request

In a previous article, I used Selenium to scrape this Excel Online file, but that felt a little indirect and slow, so here is a new attempt with new tools and knowledge gained. Full notebook at

Creating the request

In the previous article, the strategy was to scroll, find, parse, scroll, find, parse,… Now, the goal is to send requests using Python requests library to directly target the information we want.

Begin by F12 to open Developer Tools → Network Tab on Chrome, then load or F5 reload page to see a list of Network Requests being recorded, we want…

Experience the joy of human-machine cooperation

This exercise was prompted by a question on a forum regarding how to download a read-only file from excel online that required authentication to Download.

Copy pasting a few cells works fine, but Ctrl+A copy-pasting leads to just the text “Retrieving data. Wait a few seconds and try to cut or copy again.” being pasted with no data, making data analysis of the full file difficult. The follow sections will go through how to move around the document, get all the information, clean them, and put them together. Full notebook at

A journey of throwing oneself in the deep end

Photo by William Ferguson on Unsplash

— — — — — — — — Update on 15 Oct 2020 — — — — — — — — Congratulations! You are officially a Google Cloud Certified — Professional Machine Learning Engineer.

I tried a new set of 10 sample questions at
I’d say they are more difficult than 70% of the exam questions.
— — — — — ——— — — End of update — — — — —— — — — —

1 Aug 2020, I checked to see that the registration page which a week ago showed “we have sufficient beta test takers and…

A deep dive of the source to uncover import design patterns

Photo by Pascal Müller on Unsplash

In my voluntary role providing online technical support for, I come across numerous questions that allow me to dive deeper into interesting questions I usually skim through.

Today, the question is:
What’s the difference between
left_df.merge(right_df) vs pd.merge(left_df, right_df)?

The short answer is left_df.merge() calls pd.merge() .

The former is used because it allows method chaining, analogous to the %>% pipe operator in R which allows you to write and read data processing code from left to right, such as left_df.merge(right_df).merge(right_df2). If you had to do pd.merge(), this is not the chaining style but wrapping style which ends up…

3 ways to shake up your data representation and recover

Photo by Outer Digit on Unsplash

Full notebook at
Alternatively, click “view raw” at the bottom right of this scrollable frame and save the json as an .ipynb file

In this article i will explore how dataframe.stack(), dataframe.melt(), dataframe.pivot_table from pandas data manipulation library of python interact with each other in a transformation pipeline to reshape dataframes and recover the original dataframe, along with numerous caveats along the way by following along the code.

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = “all”
import pandas as pd

By default, jupyter notebooks only display the last line of every cell. The first two lines make jupyter display…

Han Qi

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store