I run a website that's in a similar situation, and it sounds like the author is close to the correct solution, but needs to take things one step further:
There needs to be a Single Point of Truth, and it should not be your little website.
BigCentralThing should be the official database that knows about where all the Whatevers are. Your little Whatever Locator site should only ever _pull_ from them. Need a new Whatever in the database? Get BigCentralThing to add it, then pull it in.
As an example, I run https://bettybeta.com/, a select guide to bouldering in Fontainebleau tailored for Women and short folk. We have a list of boulder problems in our database, but no way for anybody (even an admin) to add a new one. Every boulder problem record includes a link to its page at https://bleau.info/, who are dedicated to cataloging every climb in the forest.
Bleau.info is the Single Point of Truth. If one of our users does a First Ascent and wants to log it with us, we'll point them at that other site with instructions on how to get their climb in there, and tell them how to get us to import it. Any other path would lead to our database and the "official" database eventually diverging.
There are half a dozen other websites doing similar things to us. Some do what we do, but the ones that don't quickly find themselves out of sync and end up with a lot more work on their hands.
It's tough because, as the author says, writing the flow to import at our end is really easy (I've written it, before coming to this realization and hiding it again). But doing it is a trap.
This is partially because they import into their own system, and then export->import into OSM. At that point, they're considered an integration which imports data.
If their apps instead submitted directly to OSM (letting submission trickle back down), then the flow would fit into "user submission". Book Corners simply becomes an app though which the user performed the submission, rather than an intermediary exporting data.
>A project like OSM would be bombarded with spam and junk submissions if it didn’t have these barriers to submission.
I'm not sure about this. The barriers to submissions are honestly more likely to cause object and amenity level data to become out of date. Google is able to have such an up to date place database because it relies on a large quantity of crowdsourced submissions, and employs consensus to figure out the truth.
Google employs _engineers_ to figure out the truth. If an import pipeline is causing problems, a higher-up can just say “stop doing that or we’ll stop your pay cheque”.
OSM doesn’t have that option. If it didn’t have these guardrails in place, a million CS students will see some open-looking data (or not even that, maybe something scraped) and throw it into OSM with a cursory Python script, resulting in an almighty mess. We need the process because we don’t have Google’s leverage of “we pay you”.
Having out of date public book boxes isn't really that big of a deal, at least compared to the massive amounts of broken entries, duplicates, and incorrectly licensed data that would end up from free and unrestricted bulk imports.
As it is today you can largely expect OSM POI data to be incomplete or out of date unless you have someone particularly keen on keeping your local area up to date. OSM is best used as a background map layer for your own custom applications, like embedded maps in apps showing your own data overlayed. Rather than an alternative to the google maps app.
I wouldn't use gmap as a benchmark for up to date POIs, at least in some European countries. And I'm not talking about inaccurate opening times; but about places that closed during Covid but are still on the map.
These restrictions are unique to projects doing automated submissions of mass data.
If you're an individual just trying to make the world a better place, you can make an OSM account, go to the website, click, add a thing, hit submit, done. Someone will review it.
Apps like CoMaps and Organic Maps make it super easy to contribute data for businesses, landmarks, and such.
> you can make an OSM account, go to the website, click, add a thing, hit submit, done. Someone will review it.
I will share an unpopular opinion: thil will only make a freeworkforce for those projects who profit from using OSM. And the OSM has surprisingly unreliable data quality for walking around even in the popular tourist places like Tokyo or Kyoto, or New York. I've tried, and it's so worse than GMaps (or Bing) to being completely unusable. No work hours of Starbucks, no local dining cafe, no menu contributions.
It's great for the trail and hike outdiors, sure. But that's very different from the city map, as the nature featutes rarely change, not like city POIs.
This is kinda what StreetComplete is supposed to help with, right? it asks me questions about opening and closing hours of cafes and things.
I would assume that tourist places don't have many locals going by and updating them and tourists don't want to fill in info on that kind of app, but that's mostly an adoption question. If more people contributed more info would be mapped.
Yeah, StreetComplete helps both with adding the data in the first place and then keeping is current, since it will also ask those questions again after a while (which also gives an opportunity to check whether a feature still exists). What's less visible there is whether a cafe exists at all – it's not as good to add missing objects as it is for filling in details of existing objects, at least for the casual user.
This is trivial to navigate.
Use the notes API to post the likely presence of a public bookcase with details (link to image, description, etc); and the community will slowly ingest the contributions.
Or set up a maproulette challenge with much the same detail.
Or just a geojson dump under an ODBL licence and share with with relevant communities. The third one or something very like it is required if you have mixed ODBL with non ODBL content.
Hello folks, author of Book Corners here. I just made the whole libraries db available for download (it's OBDL licensed).
It's available from every users dashboard (it's a ~2 MB compressed GeoJSON file). It includes both libraries orignally imported from OSM and additional libraries added by users.
While I still think the whole OSM process to submit back is a bit too much for the time I have available (but I still understand and respect), at least I want to give users an additional option to retrieve the data (in addition to the public API).
If you have any other advices, I will be glad to hear them and, compatibly with my spare time, I will try to implement them (if they make sense and are useful for users) :)
> There are also important licensing questions. A user’s permission to send a library to OSM is not automatically the same as having a sufficiently clear right to release that factual information under terms compatible with OSM
which would imply that you could not make available your database available for download under ODbL. Yet you have made it available under that license.
Maybe that part of the article needs further editing?
Also, if the data of your tool is now available under ODbL, then the tasks related to importing it into OSM can be effectively performed by anyone, not necessarily you.
> Maybe that part of the article needs further editing?
Yes, I should edit it. I had not though about the bulk download until another user on Mastodon suggested it to me.
> Also, if the data of your tool is now available under ODbL, then the tasks related to importing it into OSM can be effectively performed by anyone, not necessarily you.
Definitely. I would be more than glad if libraries which are not on OSM were added back, I just don't have the required spare time to go through all the process and also maintain it.
This makes sense, but I wish there was a generally accepted way to share geospatial data and POI data against OSM IDs. e.g. I want to be able to add overlays for noise level, etc. and I find the whole thing too fragmented. The OSM way seems to be the right way, but no one seems to do it. You just maintain your own DB with an optional OSM POI entity identifier etc.
I suppose if you want something badly enough you have to do it yourself, but it's not that the thing doesn't exist though it doesn't seem to. It's that I want others to want it too :)
Can you simply write overlays as GOL files and load them from non-OSM URLs? Assuming you use the appropriate coordinate grid you won’t need POIs to layer the relevant data, and if you choose to annotate OSM’s POIs further, you can save those annotations as a personal GOL that you optionally share with others, and that also enables cross-referencing derivations if you use (unverified: trust isn’t the point, uniqueness is) signing key fingerprint as the GOL author GUID.
I've always been confused why OSM doesn't design a cheap open source stereo imager, some finetuned modern YOLO neural network could detect and blur the glass parts of windows, doorbells, nameplates and humans in the scene.
It could use GPS for rough geolocation, and 3D models of all the scenery could be generated. New contribution traces would contain changes compared to the past. Volunteers could request new paths it would like to see explored, and OSM could propose paths that cut through or ride along segments of recently submitted recordings of other users, to check if those changes are real, without checking the whole suspected recording.
Separate ground truth recording from its interpretation into mappable concepts, going out to make a recording or observation is a different task from deciding how to canonicalize the content.
Are you describing what Mapillary is doing ? It's a crowdsourced street level imagery provider (owned by Facebook since 2020, but still contributing to OSM), but they do auto-detection of feature (you can see that in the online OSM editor: Map Data > Photo Overlays > Mapillary > Map Features).
I vaguely remember a blog post about doing photogrammetry out of their images, but I'm not entirely sure about it.
Well, first step here is to demonstrate that you can do that neural network part. I think that would be a really valuable tool for more projects than openstreetmaps - google might even want to buy it.
I think the piece I’m confused most about is why the little free library website’s map isn’t the place to log little free libraries.
That being said, it makes sense that OSM has guidelines like this in place to ensure the data quality stays, well, quality.
I feel like having such a map at all is kind of antithetical to building community. Shows you don't care enough to take a stroll in your 'hood to find such a one by happenstance (or to find neighbors to populate yours) and are hunting them down to share your self-published slop or something. Besides, it makes things a hell of a lot easier for griefers and trolls.
I appreciate OSM for maintaining a higher data quality bar than other projects (Overture places are mostly junk outside of USA), but it's also just artificially limiting itself by not allowing streamlined paths to data contributions.
Like I have a list of a few hundred osm places websites that are clearly scam sites. I should be going one by one and filing them manually but I found this via a spam filter and it’s very robust. I should have a way of getting these scam sites reported to osm.
You could share it here and let more people remove the spam sites. If everybody removes a couple in no-time all will be removed.
Which is probably what might happen if you would have the option to "report them to osm". Someone should check things, because maybe you are a new person to them, maybe the spam filter might not be as robust as you might think, etc. Even if you are perfect, other submission might not be, so someone needs to check/review/etc.
It is streamlined for individuals, and I have no interest in it being streamlined for anything else.
I'd rather no data than bad data. There is no seperate baby & bath water. If a bulk source of data contains an unknown mix of good and bad data, that is all one big single item of bad data that is of no use to anyone.
I’m grateful that a principled group of people run OSM. Much like I’m grateful that a principled group run Wikipedia. But the rigidness has costs that I don’t think are being appreciated.
Mapy.com does this and I often see and resolve these notes on streetcomplete. It's nice setup, they provide a suggestion and location, it takes little effort to check if you are walking by and just reply to the note. Some enthusiastic editor (can be you back home, but it's annoying to do on a phone screen) then can make the edit and resolve the note.
Yes, the data source is clearly useful, so not having it is a loss. But an edit is large enough a privilege to require all these checks.
Perhaps it's not that OSM that needs a signal, but perhaps it's that there's an opportunity for an open map that has many signals, of which OSM and Book Corners are examples.
This can probably be added to the toolchain pretty easily when non-OSM "signals" are converted to OSM format first. All you need then is an OSM data merging tool, all other tools will then get the same data format as before.
Laying your own datasets on top of OSM is probably the most common use case. OSM isn't much of a replacement to Google and Apple maps as it is a way to build your own custom maps.
The article is definitely write or at least rewritten or heavily editorialised by an LLM. You can confirm this by skimming the author’s previous articles and compare it to this one, specially pre-2021.
Come on, the LLM language is all over the article:
> The workflow I had in mind was deliberately cautious
> The code was not the difficult part.
> These requirements are not a one-time form to complete and forget. They create an ongoing responsibility around the account, the documented process, community feedback, failures, and potential reversions.
Just a few examples. And yes, Pangram 4 also flags it as 100% LLM written. I don't mind being downvoted or flagged, but I think more people should be aware of the LLM style, even if they're ok with it being used without any disclaimer. It's honestly sad that nowadays people on HN cannot recognize this style.
Also I don't get the point why you would want them easily scrapeable to begin with.
If you have eyes to read and can walk across YOUR neighborhoods, you would have eventually seen one. I thought that was the whole point? They are not exactly hidden.
I find those public bookcase incidental low barrier offerings to a reading exprience you just stumble upon, nothing you would seek out on a map.
Why would you want to have more people from anywhere else possibly raid them for anything useful?
This way nobody is inclined to put anything worthwhile in there if it just ends up outside of their communities. Not that I'm in general pessimistic, just would like to know what is gained by putting the exact location into a map and not just have a list of neigborhoods with a book corner and optional photo on that neigborhood level, if people want to network around them.
Those people traveling to them are probably just as likely to put something in as a more local passer by. And I'm not sure why I would care that someone the next suburb over read my book. As long as someone is enjoying it, that's the whole point of donating.
Recently it's seemed like (very roughly) half the Little Free Libraries I see aren't in OSM.
I sometimes edit OSM, so I'm wondering whether any public bookshelf omissions are intentional, and adding the shelves would be unwelcome by the people running them.
I see OSM as a mapping application. It is in the real world, publicly visible for a long period of time, and there is no law preventing sharing the information (like military stuff) it can be in the mapping application (decide what is relevant enough is another story).
I would not include the wishes of the people into it, because there are various groups that might have strange ideas (dunno, like only churches of some type should be shown; or no shops of a certain kind because it hurts my feelings, etc.).
I personally would not like any edits that advances the wishes of "any" group of people (like removing features). I do not like all things that exists on maps, but I will not remove them because of my feelings.
The few places I have seen book nooks like this, they end filled up with religious literature which is usually not the intent of the person who started it.
There needs to be a Single Point of Truth, and it should not be your little website.
BigCentralThing should be the official database that knows about where all the Whatevers are. Your little Whatever Locator site should only ever _pull_ from them. Need a new Whatever in the database? Get BigCentralThing to add it, then pull it in.
As an example, I run https://bettybeta.com/, a select guide to bouldering in Fontainebleau tailored for Women and short folk. We have a list of boulder problems in our database, but no way for anybody (even an admin) to add a new one. Every boulder problem record includes a link to its page at https://bleau.info/, who are dedicated to cataloging every climb in the forest.
Bleau.info is the Single Point of Truth. If one of our users does a First Ascent and wants to log it with us, we'll point them at that other site with instructions on how to get their climb in there, and tell them how to get us to import it. Any other path would lead to our database and the "official" database eventually diverging.
There are half a dozen other websites doing similar things to us. Some do what we do, but the ones that don't quickly find themselves out of sync and end up with a lot more work on their hands.
It's tough because, as the author says, writing the flow to import at our end is really easy (I've written it, before coming to this realization and hiding it again). But doing it is a trap.
A project like OSM would be bombarded with spam and junk submissions if it didn’t have these barriers to submission. Understandable.
If their apps instead submitted directly to OSM (letting submission trickle back down), then the flow would fit into "user submission". Book Corners simply becomes an app though which the user performed the submission, rather than an intermediary exporting data.
(an import is when there isn't complete review...)
I'm not sure about this. The barriers to submissions are honestly more likely to cause object and amenity level data to become out of date. Google is able to have such an up to date place database because it relies on a large quantity of crowdsourced submissions, and employs consensus to figure out the truth.
OSM doesn’t have that option. If it didn’t have these guardrails in place, a million CS students will see some open-looking data (or not even that, maybe something scraped) and throw it into OSM with a cursory Python script, resulting in an almighty mess. We need the process because we don’t have Google’s leverage of “we pay you”.
As it is today you can largely expect OSM POI data to be incomplete or out of date unless you have someone particularly keen on keeping your local area up to date. OSM is best used as a background map layer for your own custom applications, like embedded maps in apps showing your own data overlayed. Rather than an alternative to the google maps app.
These restrictions are unique to projects doing automated submissions of mass data.
If you're an individual just trying to make the world a better place, you can make an OSM account, go to the website, click, add a thing, hit submit, done. Someone will review it.
Apps like CoMaps and Organic Maps make it super easy to contribute data for businesses, landmarks, and such.
I will share an unpopular opinion: thil will only make a freeworkforce for those projects who profit from using OSM. And the OSM has surprisingly unreliable data quality for walking around even in the popular tourist places like Tokyo or Kyoto, or New York. I've tried, and it's so worse than GMaps (or Bing) to being completely unusable. No work hours of Starbucks, no local dining cafe, no menu contributions.
It's great for the trail and hike outdiors, sure. But that's very different from the city map, as the nature featutes rarely change, not like city POIs.
I don’t think you’ll find even most the die hard OSM fan disagree with you on the POI data.
On the other hand, the road network in OSM is very high quality. As you point out, it’s used and contributed to by companies like Lyft and Amazon.
The fact that anyone can download an accurate detailed global road network for free is pretty crazy.
I would assume that tourist places don't have many locals going by and updating them and tourists don't want to fill in info on that kind of app, but that's mostly an adoption question. If more people contributed more info would be mapped.
This moves the goal posts of OSM
Or set up a maproulette challenge with much the same detail.
Or just a geojson dump under an ODBL licence and share with with relevant communities. The third one or something very like it is required if you have mixed ODBL with non ODBL content.
It's available from every users dashboard (it's a ~2 MB compressed GeoJSON file). It includes both libraries orignally imported from OSM and additional libraries added by users.
While I still think the whole OSM process to submit back is a bit too much for the time I have available (but I still understand and respect), at least I want to give users an additional option to retrieve the data (in addition to the public API).
If you have any other advices, I will be glad to hear them and, compatibly with my spare time, I will try to implement them (if they make sense and are useful for users) :)
Thanks
> There are also important licensing questions. A user’s permission to send a library to OSM is not automatically the same as having a sufficiently clear right to release that factual information under terms compatible with OSM
which would imply that you could not make available your database available for download under ODbL. Yet you have made it available under that license.
Maybe that part of the article needs further editing?
Also, if the data of your tool is now available under ODbL, then the tasks related to importing it into OSM can be effectively performed by anyone, not necessarily you.
Yes, I should edit it. I had not though about the bulk download until another user on Mastodon suggested it to me.
> Also, if the data of your tool is now available under ODbL, then the tasks related to importing it into OSM can be effectively performed by anyone, not necessarily you.
Definitely. I would be more than glad if libraries which are not on OSM were added back, I just don't have the required spare time to go through all the process and also maintain it.
I suppose if you want something badly enough you have to do it yourself, but it's not that the thing doesn't exist though it doesn't seem to. It's that I want others to want it too :)
It could use GPS for rough geolocation, and 3D models of all the scenery could be generated. New contribution traces would contain changes compared to the past. Volunteers could request new paths it would like to see explored, and OSM could propose paths that cut through or ride along segments of recently submitted recordings of other users, to check if those changes are real, without checking the whole suspected recording.
Separate ground truth recording from its interpretation into mappable concepts, going out to make a recording or observation is a different task from deciding how to canonicalize the content.
I vaguely remember a blog post about doing photogrammetry out of their images, but I'm not entirely sure about it.
This sort of feels like a generational thing but I would have flown over.
Which is probably what might happen if you would have the option to "report them to osm". Someone should check things, because maybe you are a new person to them, maybe the spam filter might not be as robust as you might think, etc. Even if you are perfect, other submission might not be, so someone needs to check/review/etc.
I'd rather no data than bad data. There is no seperate baby & bath water. If a bulk source of data contains an unknown mix of good and bad data, that is all one big single item of bad data that is of no use to anyone.
> Example of a valid and useful note can be "a new road was constructed here" or "this shop is closed and does not exist anymore".
https://wiki.openstreetmap.org/wiki/Notes
Perhaps it's not that OSM that needs a signal, but perhaps it's that there's an opportunity for an open map that has many signals, of which OSM and Book Corners are examples.
Is it possible that we (myself included) have become conditioned to think content is llm generated as the default opinion?
> The workflow I had in mind was deliberately cautious
> The code was not the difficult part.
> These requirements are not a one-time form to complete and forget. They create an ongoing responsibility around the account, the documented process, community feedback, failures, and potential reversions.
Just a few examples. And yes, Pangram 4 also flags it as 100% LLM written. I don't mind being downvoted or flagged, but I think more people should be aware of the LLM style, even if they're ok with it being used without any disclaimer. It's honestly sad that nowadays people on HN cannot recognize this style.
Why would you want to have more people from anywhere else possibly raid them for anything useful? This way nobody is inclined to put anything worthwhile in there if it just ends up outside of their communities. Not that I'm in general pessimistic, just would like to know what is gained by putting the exact location into a map and not just have a list of neigborhoods with a book corner and optional photo on that neigborhood level, if people want to network around them.
I sometimes edit OSM, so I'm wondering whether any public bookshelf omissions are intentional, and adding the shelves would be unwelcome by the people running them.
I would not include the wishes of the people into it, because there are various groups that might have strange ideas (dunno, like only churches of some type should be shown; or no shops of a certain kind because it hurts my feelings, etc.).
I personally would not like any edits that advances the wishes of "any" group of people (like removing features). I do not like all things that exists on maps, but I will not remove them because of my feelings.