PyCommonCrawl – Processing the Web

The goal of this project is to create a simple tool to process the data provided by CommonCrawl in Python.

I wanted to do something simple. The problem was that downloading all the data from CommonCrawl requires too much space (for the compressed version, something like 53TiB) and so I required a “streaming/online solution”. Fortunately, the data is divided into segments (around 56.000). So, what my solution does is to make all the process of downloading and deleting the segments transparent. It looks it if it was a single file. It is possible to iterate by line and by WARC bloc (the format for the Internet archives).

However, this solution can be extremely long, the main limiting factor being the downloading part.

All the code is avalible on Github and I created a python package.

Pyformlang

Pyformlang allows you to manipulate formal languages.

It is fully implemented in python and I tried to make the code clear enough for pedagogical purposes. The code is of course Open Source and can be found on Github. The doc is on readthedocs and pyformlang is on Pypi.

The installation is easy: pip3 install pyformlang.

Wireless sensors network for performance monitoring

This is my semester project I did at ETH in the Institute of Robotics and Intelligent Systems. My advisor was George Chatzipirpiridis.

Abstract

Sensors are getting more and more popular nowadays for performance monitoring due to the high interest in Big Data and the Internet of Things. There are now use even outside industry, in a smart home or smart cities for instance. In this project, I explored ways to build a wireless sensors network in a flexible and scalable way.

Summary

In this project, I started from the open-source project Mesh Bee which I improved to be able to match our needs. A lot of low-level features were modified or improved and at the end, the results were displayed on a website. A full description of the project can be found here. All the code source can be found on Github.

Kudly

Kudly

Story

Kudly project aims to become kids’ new best friend, but also to help their parents, providing them awesome features to keep close to their baby all day long. Using new advanced technological material, Kudly is actually the next generation of connected cuddly bears.

Starting with the morning, we decided to focus on how to make the child wake up gently. Kudly will offer the possibility for the parents to program a musical clock, and it will even be able to wake the child up using recorded parents’ voice, as if they were there.

Yet, what happen when the child wakes up earlier in the night ? When their kid will move, start crying or play with Kudly, parents will be notificated on their smartphone. This will be possible using an accelerometer and a gyroscope, an infrared presence sensor and an embedded microphone detecting whether or not the child has woken up. Imagine how nice it would be to let your children rest a few hours more with a soft music or keep him busy playing with his Kudly.

Childhood is an ephemeral moment, going through so fastly, a moment of which you want to remember. Kudly will be there to take photographs of you child and will provide you an online photo album.

How can we avoid the child to be bored by his soft toy ? Simply by making Kudly evolve at his rhythm thanks to educative programs adapted to the kid and availables online. Thus, a child who learns the alphabet or algebra will be helped by Kudly during his learning. Kudly will give instructions to the child through its speaker and will interact with him using LEDs and various sensors in its body. So what about teaching different languages or mathematics to the child?

Your child’s health is your main concern. In particular, we can think about the environment where he lives every day, like his bedroom. Data such as the ambiant temperature will be available immediately on your smartphone, thanks to Kudly’s temperature sensor.

Finally, we didn’t forget that a soft toy is done for being cuddled. Kudly is equipped with pressure sensors which allows it to know when he’s cuddled. Then, he will be able to answer and interact with the child using sweet words.

The aim of this project is to create a toy which will be as useful for children and parents. Kudly grows up at the same rhythm as the child, to offer a personalized follow-up which will delight young children as well as adults.

The code can be found on Github: https://github.com/Aunsiels/Kudly

More details: https://www.hackster.io/rose2015/kudly-22196f

BumpBand

Hi!

Today, I will show one of the project I did at Telecom Paris during my first year: BumpBand!

What is BumpBand exactly? It is a connected wristband with lights which allows you to enjoy your friend in the present moment. We realized that not much social networks put the physical link on the foreground. BumpBand creates a local and ephemeral social network putting the physical link at the first place of a relation.

How does it work? It is very easy. During an event, you have to “bump”, meaning touch someone else wristband with yours, to add him to your friend list. This list is then managed on an Android app. You can find a presentation (in french) on Youtube.

The project was made Opensource so everyone can use it. It can be found here.

In the given code, a party is implemented. One person is the administrator which represents the Bar and the dressing room.

The wristband emits light: for each event a color code can be created to make communication easier. For instance, for each drink you want to order, there can be a different color.

The wristband was done using a 3D printer so everyone can do its own.

Ciao