Nimiq Map Directory Backend - Bounty

Dear Nimiq Community, we want to build an directory for all shops that accept NIM. This directory should have an open API and be open source for others to build on. Thus, we are looking for a developer who has the time and skills to develop this project according to the following requirements:

Requirements

  1. CRUD restful API for accessing shops, supporting both online and brick and mortar
  2. Bulk Imports, e.g. via CSV
  3. Basic UI for adding and updating shops manually

The goal of the backend is make it possible to build following frontends on top of it:

  1. A map that displays locations of physical stores that accept NIM.
  2. A directory of online shops / merchants that accept NIM.
  3. Search function to easily find shops.
  4. Merchants can easily add their shops themselves.
  5. Merchants can, temporarily display additional info such as promos

Architecture and DB

As a rough architecture, we suggest building a back-end service that offers REST web services for platforms and services that want to show data from the shop directory. The service itself should then store that information in a standard SQL back-end (SQL is optional, it could also be a document based DB as long as it supports all search features), support the basic CRUD functions with minimal UIs to get started and in a later version also bulk imports.

Milestones

1. Database scheme

Should be in “vanilla SQL” so that the service can run with any SQL database
The scheme should be created in collaboration with Team Nimiq, we will define a point of
contact, could be Sven and/or also Maestro; the schema needs to acknowledge physical
shops (latitude/longitude) as well as online shops and for the latter, the reach is
important, i.e. the countries they ship to; general metadata is needed as well as what
range of goods are offered; linking shops to listings on Google Maps is just an idea here
but would be great because it would allow to automatically get metadata such as
opening hours.

2. CRUD web services

A first version can be read-only, but on the long-run, it should be possible for external
providers to add shops to the system; we think standard REST services using JSON
would be great as they are basically industry standard; we don’t have restrictions on
what technology to be used per se, as long as it’s open source and industry standard
(we want to make sure other developers can extend and modify the code).

3. Minimalistic UIs to use CRUD services

Some minimalistic UIs so that we can feed and update the system with data manually in
the beginning. Ideally, these UIs can be generated automatically from the DB schema

4. Search function

The APIs need to provide an endpoint to search for shops based on their metadata such
as location/area and range of goods offered. Could also be done later, when we have
several shops in the system.

5. Bulk import

A bulk import will be needed later this year, but there’s no rush, we would first need to
define the exact format. But in general, we expect it to be based on CSV, JSON could be
an option, too. Let’s sync on that before starting implementing anything.

The entire proposal including milestones is flexible and we would love to hear your feedback — but we would expect the resulting code to be submitted as one PR per milestone to a repository that will be provided on GitHub, but that’s not set in stone yet. If you know how, then having the entire service ready to be deployed with Docker would be a plus. While we think that SQL and REST are the best technology choices for this implementation, we’re also open to considering a GraphQL endpoint or using a document-oriented database instead of SQL.

Budget

$3000 in NIM or BTC

Timeline

2 months

[Edit Mar 21, 2022: The first version has been delivered by @Stefan today. He shared the work with a friend of his and they will now share the reward of $3000! Thank you very much for the work!! The code can be found here. We will now be working on making the API available for the public.]

5 Likes

Interesting bounty! I have a couple of questions:

  • What authentication / authorization should be in place?
  • How many shops are estimated? Could determine if DB search is enough or if a more advanced search solution is required.
  • Would it be ok to use a DB abstraction layer that supports multiple SQL variants?

What’s the process for the bounty? First to build wins?

@Richy Has this been completed yet?

I don’t think this bounty makes sense because:

  1. Supabase has all these features
  2. Most of these merchants will be using salamatex or similar, we should sync the data with these services (which could be done with a simple script that connects to supabase)

Amazing! I’ll tell @Micha and @svub to take a look and get in touch with you.

1 Like

I updated the frontend and backend, it now shows image urls and shipping countries from the database. You can also see a demo of the full text search: nimiq-shop-directory - nimiq-shop-directory

Three things I am considering:

  1. Maybe there should be an option to include or exclude shipping countries. In other words, set the countries that are shipped to to be “the following countries” or “all countries except these.” This could be a numeric 1/0 value stored in a seperate column, or shippingCountries could become a JSON data type.
  2. As I mentioned earlier, even though it isn’t part of a bounty, maybe we should be pulling data from yelp, salamatex, schema.org json-ld, etc. instead of aggregating the data ourselves
  3. What data should we allow merchants to put in their promos? Should it just be text or should there also be values like percentSaved, amountSaved, promoCode, color, lastsUntil, etc?

Note that I still need to export the SQL/Supabase config before the project can be considered complete, I’d like to get the schema approved by a dev before I do so.

Also note, I built the frontend in Vuetify, if you want to use it I have some code (CSS and config, not 100% done yet) to help make it match nimiq style.

Any news from the devs? I haven’t heard from them yet.

Hello @code3z! I checked out the app and all it’s code, thank you for working on the idea. I must point out though, that a specific requirement was to have an encapsulated REST web services that then can deployed as part of our infrastructure, thus also the hint with Docker.
I mean your prototype is pretty cool and I’m impressed how easy it is with Supabase to pull up something like that – that’s really good to know for future MVPs and prototypes.
In the meantime though, @Stefan has been working on a server-side implementation using Laravel, the code is here https://github.com/Eligioo/nimiq-shop-directory-backend
Would you be interested supporting this effort by contributing, reviewing the code and giving feedback?

About the data aggregation: yes, that’s the plan, that’s what the bulk import functionality should be for. We already have two data providers and most likely more coming next year.

Hi, thanks for checking it out. I totally get that this is not enough to earn the bounty, and I did not expect it to be. I have no experience with Docker and very little experience with SQL, but I will try to make a docker image for this.

The link to the nimiq-shop-directory-backend is down. When I started this, I already knew about that repo, but I worked on this anyways b/c it didn’t make sense to me to implement yet another CRUD app from scratch in PHP and the project or any of its branches hadn’t been updated for several months (since september).

For data aggregation, I would suggest including two columns in the table: one to tell which data provider the entry came from, and another to tell what ID the row has in the data provider’s database so it can be updated.

@svub Hi, I was having some trouble getting docker on my computer. Today I used gitpod instead and I think I am done or almost done. Just to be clear, supabase is open source and CAN be run on your own servers (even the GUI!).

You simply have to clone the project and run the commands on the README.

Hi, everything listed in the requirements is done except the bulk import functionality.

Bulk Imports, e.g. via CSV

Could you provide more details on this? Does it have to be CSV? You can already use JSON arrays to bulk-import via the api (e.g. thru curl), is that enough? Or does there need to be a GUI? Is it okay if the service key (bypass security) needs to be pasted into the web UI for the bulk import to work? Or would you prefer a command-line utility instead? Thank you!

Note that I’m also willing to add some more features not included in the bounty description (like image uploads) and help maintain the code, but only after I receive the bounty.

Hi, I’m afraid my previous message was not written clearly enough, we are going for the implementation happening here: https://github.com/Eligioo/nimiq-shop-directory-backend - please check it out and give feedback.

Ok, I don’t see anything there, is it a private repo?

2 Likes

@Stefan would you mind making it public already?

Here you go:

2 Likes

Woohoo! :partying_face:

Thank you @Stefan! With making the repo public the mission is accomplished and the reward is on it’s way to you! We will now be working on making the API available to the public. Stay tuned. :slight_smile:

2 Likes