Generate randoms, more than numbers

  • In the landing page of your website, you want to have the picture of your customers right by their testimonial
  • You are looking for a cover to use on a piece of music
  • A random picture to use for an Instagram post
  • For your profile picture, you would like to use a random anime character
  • For your new project, you’re looking for a cool new name

All these questions can be answered using “generative adversarial network” (GAN) which is used to create fake versions of almost anything.

Visit This X Does Not Exist for collection of these websites.

Recursive asynchronous function call

One of the well-known Redis clients for NodeJS is ioredis. When I was trying to work with Redis streams using this library, I used the example they provide in their GitHub page. When I read the code, I noticed in order to constantly listen for new messages, they have used a recursive call to an asynchronous function:

async function listenForMessage(lastId = "$") {
  const results = await redis.xread("block", 0, "STREAMS", "mystream", lastId);
  const [key, messages] = results[0]; // `key` equals to "mystream"

  messages.forEach(processMessage);

  // Pass the last id of the results to the next round.
  await listenForMessage(messages[messages.length - 1][0]);
}

listenForMessage();

The XREAD function blocks the execution, until we receive another message in the stream; Then it recalls itself to get another message on the stream.

My question was if this is a safe approach to iterate over the message we receive; Specifically, I was interested to know whether we’ll have stack overflow over time, or some mechanism like tail call optimization will take care of the growing stack.

To check this, I simply logged the stack after each call: console.trace()

And then I ran the code and started adding items to the Redis stream in one second intervals:
redis-cli -r -1 -i 0.2 xadd mystream '*' item 1

In this way you can see the stack trace, and it becomes obvious that function calls pile up; Every function call will wait for the return value of the previous one, and it will lead to memory leak.

By the way, don’t forget to set stack trace limit when running the program, otherwise you’ll see only 10 frames: --stack_trace_limit=200

How to solve it?

Just don’t await on the function call! You don’t need the result and by letting the current execution to be finished, the stack frame can be released.

SQL and Asking Questions

What’s the difference between relational and NoSQL databases and when do you choose one over the other?

Strangely, not many developers can give you some comprehensive and tangible answer for this question. I’m not going to delve into the details, because there’s already so many places that explain the differences (mainly data model, data structure, scaling and development model).

I just decided to write this post because right now I found another very compelling reason for using relational database over document based databases when I was listening to “The Change log” podcast, episode “What’s so exciting about Postgres?

A Review of Postgres version 12 - Performance Monitoring - Blogs - Quest  Community

Analytics! Or more simply, asking questions. Imagine you’ve got tons of data and now you want to find out about “How many users signed up during the last week?” If you’re using a document db, now you’ve got to go and traverse through those documents three layers deep, something that might not be very much easy.

On the other hand, this is just so darn easy in relational databases, it’s actually what SQL was designed to do. This might not be something you need right when you start your project, but as it gets bigger, you definitely want to ask more and more these sorts of questions.

It’s not that you can’t do these in NoSQL, but both the ease and also the efficiency makes relational databases the clear winner when it comes to analytics.

How to write your first Firefox add-on?

How to write your first Firefox add-on?

Due to U.S. trade controls law restrictions, your GitHub account has been restricted.

This is the message that’s been recently sent to Iranian developers from GitHub, stating their accounts has been restricted.

The political reasons and consequences of this action is not what I’m going to talk about (this is not the right place), Instead I’m gonna focus on what happened after this announcement to GitHub site: A horrendous yellow banner, screaming our misfortunes on top of every GitHub page:

github

So I thought this might be a good time to try writing my first add-on for Firefox. It seemed like a trivial task and I was curious how much it’s going to take my time. Simple tasks must be simple to do.

At first I tried to remove the banner in browser inspector. All I had to do was to set display property to none for the element and it was gone.

So, that’s what we want our add-on to do: To simply set a CSS property for an element.

Write the extension

MDN web docs has a page called Your first extension and I followed the steps it said:

At first I created a directory for my add-on.

Then I created manifest.json file. This file is the skeleton of your extension; What files does it include, a short description, extension version, icons and everything else.

content_scripts section is the most interesting part: You define in what domains, which scripts must be injected.

I wanted to modify appearance of GitHub, which means all I wanted to do was to add a CSS property. So I modified content_scripts in manifest.json to look like this:

"content_scripts": [
  {
    "matches": ["*://*.github.com/*"],
    "css": ["github-warn.css"],
  }

And in github-warn.css I wrote this:

div.position-relative.js-header-wrapper > div.js-notice.flash-warn {
    display: none
}

Run and debug

Now let’s test the add-on. First we must load it into the browser. To do so go to about:debugging in FireFox and click on “Load Temporary add-on”. By selecting a file in your add-on directory it will be loaded in FireFox.

And now when we go to GitHub we no longer see the banner.

But there’s a problem: It seems it take a little bit of time before we see our CSS effects in browser. When you open the site you can see the banner when page is being loaded and after a few seconds it disappears.

To solve this issue we have to change when the CSS get’s injected. It can be altered by run_at property in content_scripts: "run_at" : "document_start"

It worth to take a glance of content_scripts parameters in here.

Publish

There are two options for add-on publishing:

  • Self publish: You will be given the signed .xpi file that you can distribute yourself.

  • Publishing on FireFox Add-on’s page (AMO)

In either cases first you have to login to Mozilla Developer Hub and upload your zipped add-on directory. To do so, go to your add-on directory and run:

zip -r github-warn.zip . 

Tip: If you’re on macOS, to avoid warnings make sure .DS_Store and __MACOSX are not included in zip file. You can delete them by executing this command:

zip -d github-warn.zip __MACOSX/* .DS_Store

After you’ve uploaded the zip file you can choose between self-publishing and publishing on AMO. In case you choose the latter, it might take about a day or two before your extensions be approved and becomes publicly available. Self

You can access the add-on code explained in this post on GitHub.

Trie implementation as Redis module

In this post I’m going to explain how did I implemented Trie data structure in C++ and imported that as a new data type into Redis.

I start with the implementation of Trie in C++ (who did I get started and how I tested the code), and then went into the details of introducing the Trie as a new data type to Redis. (By the way, there’s already a Trie module implementation in C available which you can checkout in here.)

Trie data structure

GeeksForGeeks has proven over years that it’s the best source to learn or refresh my mind on all sorts of algorithmic problems. So I head over to it and as I was going forward I also started the implementation. After a couple of minutes I had insert and search functionalities.

Test it!

It’s was a long time since I had my hands on C++ codes and as a TDD fanatic I had to have automated tests for my Trie.

After a brief search I found three candidates:

  1. Google tests
  2. Boost.Test
  3. Catch2

The first two they both mature and they were filled with all sorts of features, which meant they had their own subtleties. Also considering the size of my tiny project they were somehow huge. I wanted something simple and clean, so I can just drop it in my project and use it; And there it comes the Catch2. It came to my attention after seeing it mentioned on SO and Reddit.

A modern, C++-native, header-only, test framework for unit-tests, TDD and BDD

It’s header only, simple to use and documentation is clear. You just put the header file in your project and reference it in your test file. I could use it immediately. Now I had my automated tests.

Trie.remove

Implementation of Insert and search was fairly simple; remove was a little bit more interesting. The reason was that when there’s no more keys in the node you have to delete the node. Then you have to move up and check the same thing for the node’s parent, because if it was created just for that specific key, the delete operation must take place over and over until you reach the root.

Note: One thing that I have not implemented yet and would be good to have, is to return a value denoting whether key was deleted from the Trie or no (Which means it didn’t exist in the first place).

Edge cases

Just make sure you’re giving proper index number to characters you want to save in Trie; Numbers, uppercase letters and punctuation marks.

Redis modules

First when I was asked for writing a Redis module I thought of Lua scripts made it possible to write extensions for Redis. Redis modules are totally different beast. These are C shared libraries that you can load on runtime or startup. They let you define new commands or new types in Redis core. To know the exact differences I suggest reading this blog post RedisLab site.

This post also is the perfect place to start writing your first module. Follow the steps and you have your new Redis command, HGETSET; A dummy commands which let’s you get and set a key at the same time. Fortunately compiling Redis modules requires no special linking. Just add redismodule.h to the project and you’re ready to go.

Define new types

RedisLab sample, which is the same as Redis Module SDK sample program, they both show how to add a new command to Redis and none of them define a new type. Delving into documentation with no concrete example was a futile effort for me, so I looked for code samples. The best one I found was the HelloType sample in Redis repository. The other one was another implementation of Trie in C. By looking at the code and visiting documentations you could easily figure out how to manage memory and allocate space to your data type.

C vs. C++

Redis modules are supposed to be written in C and loaded as shared libraries. In order to use C++ for writing modules all you have to do is to add extern "C" at the beginning of RedisModule_OnLoad to make it visible to Redis.

Code

You can checkout the final result in this Github repository.

Essential skills that we didn’t learn in university

It’s been about a year that I’m monkey coding in a some-how large scale company and getting my head around real coding. In a year that I spend in industry I found some of most important tools and concepts that I use in my every day job, are the the ones that I didn’t heard a word about them in school. Here’s the three most important God-knows-why-they-do-not-teach-us-in-university things that are at top of my head :tdd2

First and foremost is TDD : I can’t even remember how did I wrote programs without unit tests, I literally can’t. I mean did we just open up an editor and start writing code ? How did we change the code ? How were we confident about it working as expected ? All we learned was waterfall design and UML. Heaven forbid !

Second one is Version control : It’s no longer about code bases of two or three hundreds lines of throw-away code that you would normally do for your final projects in university, We’re now talking about thousands lines of code. Version control is the only way that you cope with immense amount of complexity in large code bases and it also helps me to take baby steps toward a solution.

vcs

refactoringAnd finally the third one, Refactoring : In university what matters is you can provide expected behavior, before project deadline. After that your code gonna be worthless. No one else will read it, you’re not expected to add more features and who gives a damn about bugs ? ( They just don’t have to pop up during presentation )
For me refactoring is synonym to responsibility. Your code is your responsibility and you have to take good care of it.

Path Navigator

A couple of month ago I started a tiny program as a hobby that would let you to easily open new directories and navigate between them without moving your hand off your keyboard. It was an attempt to eliminate mouse in this process, yet making it real simple by giving auto-completion. Kind of like tab-completion in Bash, but now for Windows Explorer.
I’m using it every day, it works seamless and although it seems quite naive, but it has become a must-have in my toolbox ( I can’t stress enough how much I loathe mouse ! ). A few days ago I spend time to polish it a little and republish it.

A few bug are fixed and UI has changed dramatically ( although there’s barely such a thing as UI in this program ). Borders removed and you can choose to have Van goghs’ starry night as background of the windows 🙂

navigator
As opposed to always, I decided not to upload it in Ge.tt and instead put it in my Dropbox and give links to the directory that contains the program. In this way I can easily update files without invalidating links.
Source codes are available in same directory and you can download them. Maybe in future I’ll put it under version control on github of source-forge.

PS : There’s a golden tip for using this program, and that combining it with Clover. Clover is a program that makes default windows explorer like Chrome browser by giving you multiple tabs in same window. Whether you use Path navigator ( or as a like to call it, Panvig ) or not, I encourge you to download Clover.clover

Editing Code whilst preserving its formatting

Earlier in one of my posts, I explained how can you reformat your code using NRefactory library. But, as matter of fact what you really want in your day to day task is not reformatting code, au contraire, you want its formatting to be preserved. You don’t want to commit dozens of files to your repository merely because their formatting’s changed !

Now this question arises that whether is this quest possible in NRefactory or not ? Well of course it is, and it’s really easy. ( Once you know what you have to do, who you have to call and … ) BTW, NRefactory is an essential part of SharpDevelop IDE and we don’t want our idea to reformat our code every time we use intellisens.

NRefactory has a class that’s dedicated to do such and it’s called DocumentScript.

Document scripts works by manipulating IDocument objects, which are specialization of string builder class. This class gives you this opportunity to change your document based on text location that were already stored in syntax tree. It also keeps track of latest modification and map them to current document, so you don’t have to worry about whether multiple changes on the same document will be applied correctly or not.

IDocument document = new StringBuilderDocument("Your code source");
CSharpFormattingOptions policy = FormattingOptionsFactory.CreateAllman();
var options = new TextEditorOptions();

var script = new DocumentScript(document, policy, options);

Now you can use methods like Replace, Remove, scriptInsertAfter and etc. on you AST. It works with both AST nodes as well as offsets.

Beside that, there are some predefined methods on

script.ChangeModifier(declaration, Modifiers.Public);

What Wronge With RSS Readers

I’m using Feedly on my tablet on a daily basis for rsschecking weblogs, and I use Pocket to read them later.
Well, they’re quite fast, you don’t have to tolerate all those annoying ads, size of the contents to be downloaded decreases dramatically ( very important factor when you’re limited to 4GB per month! ), and their much more legible.

But there’s something that really bugs me : When you interact with websites and weblogs in this way, you can’t socialize with other people that are reading the same content as you do. You can’t see comments or post any. I mean, who bothers to reopen the site in browser to just see comments or post one ?
The spirit of these readers are literally in contrast with what that every other app and program and company on the planet is investing on, connecting people with same taste and interests to each other.
ywnPPK7For instance, although I hate Google plus interface ( and also Google plus itself ), but still I find it much more rewarding and enjoyable to follow news and people with it. It gives you this chance to be part of gathering of all people who care about the same thing as you.

IMHO, it’s important that you have this ability to exchange and communicate with source website. Implementing this feature and  making it accessible can really be a big improvement among RSS readers. I assume the basic infrastructure for this capability already exist, So what are they waiting for ? Those guys really need to shake the dust off of their products !

PS : I must admit it’s quite a controversial topic. For example I’d like to refer to this “I suggest you …” page for Feedly.

Binary Operator Expressions

Suppose in a hypothetical scenario you want to reverse all binary operation in your file, and also you want them to be fully parenthesized. What would you do ?
One solution is using recursion which might be a little messy an unclear. Another solution to this problem can be achieved by taking advantage of binary operator expressions in NRefactory.
Fortunately with NRefactory this operation is rather a trivial task. A little tricky, but still simple.
You can use this feature to process any kind of binary operation and interpret it the way you like. ( like generating code )

First, let’s create a test fixture for it ( Hooray ! Now you know what TDD is : )

[Test]
public void BinaryVisitorTest()
{
	string input = " a > b || ( x > y && y > z )";
	var parser = new CSharpParser();
	Expression expression = parser.ParseExpression(input);

	var visitor = new BinaryOpVisitor();
	expression.AcceptVisitor(visitor);
}

Since we’re interested in visiting an expression we don’t bother by parsing a whole C# file, we give the parser our expression as text, and it gives use the expression as AST.
In next step, we create a visitor class and visit binary and identifier expressions :

public class BinaryOpVisitor : DepthFirstAstVisitor
{
	public override void VisitBinaryOperatorExpression(BinaryOperatorExpression binaryOperatorExpression)
	{
		Console.Write("(");
		binaryOperatorExpression.Right.AcceptVisitor(this);
		Console.Write(binaryOperatorExpression.OperatorToken);
		binaryOperatorExpression.Left.AcceptVisitor(this);
		Console.Write(")");
	}

	public override void VisitIdentifierExpression(IdentifierExpression identifierExpression)
	{
		Console.Write(identifierExpression.Identifier);
		base.VisitIdentifierExpression(identifierExpression);
	}
}

Be careful as opposed to what we do regularly in node visiting ( or actually what depth first visiting is all about ), we didn’t call base visitor in binary operator expression visitor.

Now, for this special case, the output will be :

(((z>y)&&(y>x))||(b>a))

Easy peasy !

For readers with data structure background I’d like to mention that what we did in here was actually an in-order visit of the nodes in AST. You can simply turn it into post/pre-order by putting base class call before/after your code in the visitor.