Journal archives for April 2019

April 10, 2019

iPhone uplodder

https://www.inaturalist.org/observations/15613866 iPhone camera with voorzetlens
https://www.inaturalist.org/oauth/applications
https://forum.inaturalist.org/t/standalone-uploader/1742
I am thinking a combination of the existing iPhone uploader and the website uploader. Organize the photos, add the ID and annotations, then either upload a batch of observations or upload all of them. This seems like it could result in quicker times to organize photos and a much lower risk of a crash losing the work. Upload times would of course be the same, but it would be possible to just have it run in the background.

I can confirm this - other than uploading the photos, the main bottleneck seems to be loading the metadata. I found that I had to limit my batches to about 25 observations to make the process workable. But even then, I still found it was taking far longer than it really should do, and every now and I would have to abort it due the browser locking up.
https://www.inaturalist.org/pages/api+reference
So in the end, I went down the same route as @glmory and wrote my own uploader. It also uses python, but not pyinaturalist, since I wanted more control over how the inat apis 1 are used. Doing things this way dramatically reduces the total upload time, and I can just run it in the background while I do other things. It also allows me to add annotations at the same time and simplifies the process of grouping multiple photos.
https://forum.inaturalist.org/t/standalone-uploader/1742
Even if the issue with loading metadata was somehow fixed, I doubt whether I would want to use the web interface for uploading again. The only major thing missing from the current apis seems to be ID suggestions using inat’s computer vision - but I very rarely use that, so that’s not a big deal for me.
https://www.inaturalist.org/pages/computer_vision_demo

That seems unusually excessive - what sort of internet connection do you have? It might be worth trying some of the online speed-testers to see what your upload speed is like (download speed is less relevant). And what is the average size (in megabytes) of the photos you are uploading? Do you crop them first?

My uploader is part of a bigger program at the moment so I’m not able to share it. But in any case, I don’t think it would help you much if you don’t have a reasonably good internet connection.
= = = = = = = = = = = = = = = =
The system has to resize images that are too large, and that will contribute greatly to the processing time of an upload. You could test it by doing a similar size batch but with the photos all pre-scaled to the iNat maximums. I know my camera is set to take photos that are nearly double in size (or 4x the qty of pixels!) to the iNat max. But even then, mine would not take more than 30 mins tops to upload 100 obs with approx 200 photos.

I think bottlenecks are going to be (in order of impact):
https://www.inaturalist.org/oauth/applications
upload speed
activity on the site at time of upload
image size
I think the upload might be staggered, meaning that it doesn’t all get processed at once and the site come to a grinding halt for everyone else while it processes. This might explain why an external uploader might be getting faster results. If this is the case, do we really want large processing volume burdens being developed? It’s fine for occassional situations, but if the number of people using it grows, then you might be getting faster processing of your uploads, but causing unacceptable delays in other parts of the system such as in just loading observation views! I would encourage you to contact the developers to make sure your impact will not be detrimental.
https://www.inaturalist.org/oauth/applications

If you have questions about how iNat works, please just ask. Regarding the website uploader, we do not resize images in the client. Well, actually we do, but only to make a thumbnail to upload for vision suggestions. The final cutdowns happen on the server, and yes, there are performance implications for the site as a whole. Improving server-side image processing is definitely on our radar, though frankly, it only creates serious problems during periods of extreme usage, like the CNC or the Penang Incident. A third party client that cuts everything down to 2048x2048 before upload might feel a bit faster due to reduced server-side processing time… or it might be faster b/c you’re uploading less data. We also only upload a max of 3 observations at a time in the website uploader, so if a 3rd party uploader creates more simultaneous upload requests it might get the whole job done faster, but could theoretically function as a Denial of Service attack if it swamped all our server processes.

I don’t know why the uploader is so slow for some people. I’m sure we all have our theories, but what would help us address the problem are concrete examples including excruciating amounts of contextual info: what browser you’re using, what your internet connection speed is like, specifically what files you’re trying to upload, etc. Some issues might be solved by a third party uploader, but not things like connection issues. While diagnoses might be helpful, reproducible examples will always be more helpful. There are a lot of variables.

Finally, we’re not going to make a standalone uploader, or at least not any time soon. We’ve toyed with the idea of a desktop app a lot, but ultimately, of the many potential uses for such an application (offline use, backups, etc.), faster image upload has never been on our radar. If you have the bandwidth to upload and the website’s not working, we should fix the website, not make more software that works better.

For folks interested in continuing to talk about standalone uploaders, using the API, or if you have other questions about rules around 3rd party apps feel free to check the documentation, message help@inaturalist.org, or ask/discuss in the General category after reading up at:

https://www.inaturalist.org/pages/developers
https://api.inaturalist.org/v1/docs/
https://www.inaturalist.org/pages/terms

*and, please remember to keep discussions on topic. If you want to have a one on one unrelated discussion, you can always directly message someone privately or start a new topic.

= = = =
OK, here is how you can see your options: In the Community ID box, click "compare", then set the Place to "South America", and for Taxon, type in Calycopis, you'll get: these 4 matches in that genus: C.origo, C.isobeon, C.caulonia, C.cerata. But then try "Lycaenidae" (the larger family), and you'll get Ministrymon azia, and Strymon melinus added to that list of 6 very similar species. You may need a more local expert.
https://www.inaturalist.org/observations/identotron?observation_id=9344400&taxon=47157#establishment_means=&order=&place=97389&taxon=47923
= ==
It is possible using the unobserved_by_user_Id parameter in the observations view.

2 caveats

if you do a species view, you are restricted to the 500 most frequently seen species unless you start filtering down (ie add birds or beetles etc)
the ‘unseen’ is not geographic specific, it counts you as having seen it if you have seen it, but not in the location you specify.(so for example in the list below, Grey Kingbird will not appear, even though I have never observed one in Ontario, just elsewhere)
For excample this url shows all the birds reported from Ontario that I personally have not seen (or more accurately not recorded an iNat record for as I have actually seen some of them)

https://www.inaturalist.org/observations?place_id=6883&subview=table&taxon_id=3&unobserved_by_user_id=cmcheatle&view=species 9

you can add additional filters for hrank and lrank equals species if you want to get rid of the genus level records.

In addition to the unobserved_by_user_id search filter for the website, you can try out the “Missions” feature on the Android app.

Posted on April 10, 2019 10:14 by ahospers ahospers | 0 comments | Leave a comment

Cameras for Nature shots

https://forum.inaturalist.org/t/good-cameras-for-nature-shots/1064

https://forum.inaturalist.org/t/focus-stacking-with-olympus-tough-tg-5/1973/4

Trends blog (Christian Rudder maybe) did an interesting analysis of cameras that produced the most attractive photos for online dating purposes (see https://theblog.okcupid.com/dont-be-ugly-by-accident-b378f261dea4 4). i wonder if there’s any sort of similar analysis for recent nature photo cameras?

https://forum.inaturalist.org/t/good-cameras-for-nature-shots/1064
samsung SM-G950U 1303 117
samsung SM-G960U 680 87
Google Pixel 2 713 79
samsung SM-G930V 864 79
NIKON COOLPIX P900 1026 65
samsung SM-G930F 281 52
Canon Canon EOS 7D Mark II 1086 50
samsung SM-G965U 441 46
Canon Canon PowerShot SX60 HS 856 43
Google Pixel 2 XL 493 41
Canon Canon PowerShot SX50 HS 628 36
OLYMPUS CORPORATION TG-5 564 36
samsung SM-N950U 271 35
motorola Moto G (5) Plus 344 34
Google Pixel 3 557 34
samsung SM-G955U 149 34
Apple iPhone 7 293 30
Google Pixel 693 30
NIKON CORPORATION NIKON D7100 321 29
samsung SM-G892A 397 26
Canon Canon EOS Rebel T6 237 26
samsung SM-G950F 179 25
OLYMPUS CORPORATION TG-4 521 25
NIKON CORPORATION NIKON D3400 269 24
samsung SM-A520F 206 24
NIKON CORPORATION NIKON D500 249 24
motorola moto g(6) 137 24
Canon Canon EOS 7D 311 24
Apple iPhone 6 186 22
SONY ILCE-6000 193 22
samsung SAMSUNG-SM-G930A 108 22
motorola moto x4 165 22
samsung SM-N960U 192 22

Camera Usage Sample
https://docs.google.com/spreadsheets/d/1fliiB5RgbYx9-N88R-xPMsUfDVjfnLLo9p_tKERKJrQ/edit?usp=sharing
https://theblog.okcupid.com/dont-be-ugly-by-accident-b378f261dea4


https://forum.inaturalist.org/t/focus-stacking-with-olympus-tough-tg-5/1973/4
BTW, the Olympus Tough TG-5 also has GPS, and geotags a photo’s coordinates in the metadata (exif ). This data is seen and used by iNaturalist for the location. It can sometimes take a half a minute to lock on, and it helps if you download the GPS Assist Data to the camera beforehand

= = = =
Most of my extreme macro of ants, spiders, and other arthropods not on slides is done with a Mitutoyo 5X microscope objective which has been attached to my camera. To take the photos I use a Stackshot which moves the camera and takes photos. For arthropods on slides I am now using a Nikon LaboPhot microscope. Generally I put my camera on silent shutter and continuously take photos as I slowly focus through. In both cases I stack the photos in Zerene Stacker.

I have a bunch of old records from Calflora:
http://www.calflora.org/cgi-bin/contrib_profile.cgi?seq=104142

= = = =
OK, here is how you can see your options: In the Community ID box, click "compare", then set the Place to "South America", and for Taxon, type in Calycopis, you'll get: these 4 matches in that genus: C.origo, C.isobeon, C.caulonia, C.cerata. But then try "Lycaenidae" (the larger family), and you'll get Ministrymon azia, and Strymon melinus added to that list of 6 very similar species. You may need a more local expert.
https://www.inaturalist.org/observations/identotron?observation_id=9344400&taxon=47157#establishment_means=&order=&place=97389&taxon=47923
= ==

Posted on April 10, 2019 12:20 by ahospers ahospers | 0 comments | Leave a comment

April 24, 2019

Search FAQ Filter Cribsheet

To find identifications by a specific user:
ident_user_id=12345

see - map of the last five minutes: https://simonrolph.github.io/inatcounter/

How would I put the pieces together to get a report of all taxa that do not have photos assigned? I messed around with it a bit but couldn't make it work. Thanks!

https://www.inaturalist.org/journal/tonyrebelo/17377
Some shortcuts to filtering data
(this does not include simple terms accessible in the grey filter box: explore there)

TEXT Searching comments:
all:
https://www.inaturalist.org/comments?q=silky%20nest
your own comments:
https://www.inaturalist.org/comments?for_me=true&q=silkypuff
also:
https://www.inaturalist.org/comments?commit=Search&q=robin 1

Searching Identifications:
maverick IDs:
https://www.inaturalist.org/identifications?taxon_id=119245&category=maverick
IDs not current:
https://www.inaturalist.org/identifications?taxon_id=119245&current=false
e.g. observations that are currently falsely identified by me relative to community ID.
https://www.inaturalist.org/identifications?current=true&user_id=tonyrebelo&category=maverick
Inactive taxa:
https://www.inaturalist.org/taxa/search?is_active=anyutf8=%E2%9C%93&view=grid

Useful extra terms to add to filters in filter bar:

(for multiple taxa, users, projects, fields separate values with a comma, note "ids" not "id")
&taxon_ids=12345,67890 &geoprivacy=open,obscured

Photos:
With & without: &photos=true or &photos= or &has=photos ; &photos=false

Observations without any ID:
&iconic_taxa=unknown

Observations identified by a particular user (e.g. tony_rebelo):
&ident_user_id=tony_rebelo

To exclude:
&without_taxon_id=1234
&not_in_project=redlist-s-afr
&not_in_place=1234

Specials:
?not_matching_project_rules_for=redlist-s-afr&project_id=redlist-s-afr

Observation Fields:
with a field:
&field:Habitat%20(s%20Afr)
with a field and value:
&field:Habitat%20(s%20Afr)=Nama%20Karoo

Observation field view:
https://www.inaturalist.org/observation_fields/1234
Observation field view for a value: (e.g. Fynbos in Habitats-s-afr):
https://www.inaturalist.org/observation_fields/7498?value=Fynbos (case sensitive)

Annotations:
Plant phenology:
&term_id=12
Sex:
&term_id=9 (male: &term_value_id=11 female: &term_value_id=10)
Life stage:
&term_id=1 (juvenile: &term_value_id=6)

OBSCURED DATA: (observations and identify)
no positional data: https://www.inaturalist.org/observations?acc=false
above: https://www.inaturalist.org/observations?acc_above=10000
below: https://www.inaturalist.org/observations?acc_below=3

user private: https://www.inaturalist.org/observations?geoprivacy=private
user obscured: https://www.inaturalist.org/observations?geoprivacy=obscured
user obscured or private: https://www.inaturalist.org/observations?geoprivacy=obscured_private
taxon private: https://www.inaturalist.org/observations?taxon_geoprivacy=private
taxon obscured: https://www.inaturalist.org/observations?user_id=kueda&taxon_geoprivacy=obscured

LISTS:
?list_id=1471294 (e.g. https://www.inaturalist.org/lists/1471294-Trees-of-South-Africas-Check-List)
trees: https://www.inaturalist.org/observations?list_id=1471294

IDENTIFY:
exclude a taxon:
&without_taxon_id=
e.g. Moths
https://www.inaturalist.org/observations/identify?verifiable=true&taxon_id=47157&without_taxon_id=47224

FLAGS
via flags: https://www.inaturalist.org/flags?taxon_id=129714
via taxa: https://www.inaturalist.org/taxa/129714/flags
query inactive: https://www.inaturalist.org/taxa/search?is_active=any&q=corycium&utf8=%E2%9C%93

PROJECTS:
projects in a place: (restricted to those with a centroid in the place)
https://www.inaturalist.org/projects/browse?place_id=botswana

TAXON PAGES:
center map and zoom: #1 out #9 close, latitude, longitude
#6/1.318/32.036
e.g. https://www.inaturalist.org/taxa/map?taxa=74191,74190,68137,74189,74188,43523#7/5.791/41.797

TABS: opening pages on special tabs:
for Places add: #observationstab #peoplestab
for Taxa add:
- also /map#2/0/0
for Observations add: ?view=species view=observers

Your statistics:
https://www.inaturalist.org/stats/2018/you

To find a value of a project or field or user: add ".json" to the url, or use chrome > inspect
using APIs: http://api.inaturalist.org/v1/docs/#!/Observations/get_observations

https://www.inaturalist.org/journal/tonyrebelo/17377

Counts Pages... naamgeving iNaturalist gebruikt WikiData
https://www.wikidata.org/wiki/Wikidata:Introduction
https://www.beeldengeluid.nl/kennis/blog/deel-2-het-wat-en-hoe-van-wikidata
https://tools.wmflabs.org/pageviews/?project=nl.wikipedia.org&platform=all-access&agent=user&start=2018-01&end=2019-04&pages=INaturalist|Waarneming.nl|Waarnemingen.be|Observado.org

Posted on April 24, 2019 17:13 by ahospers ahospers | 1 comments | Leave a comment

April 16, 2019

create iNat observation I open Google Photos, press drag across the pix I want to upload observation, click on share icon,click on the iNaturalist icon.

While that is a good way to get a camera app to record more accurate coordinates, it shouldn’t be any different than using iNat and waiting for the accuracy to get down to an acceptable level. Theoretically iNat’s doing exactly the same thing as the Google Maps app: waiting for the operating system to provide better information as it triangulates from cell towers and receives better signals from more GPS satellites. The main difference is that iNat will stop once it hits what it think is an appropriate level (I think it’s 5m or something).

DSLR and Garmin Dakota 20. Then I use Nikon’s ViewNX-i software to do the same as Lightroom does

I almost always use my Google Pixel 2XL stock phone cam, which automatically records the GPS coordinates and works seamlessly with Google Maps and Google Photos. In the stock camera settings one must toggle on “Save locations.” In my Google Photos app settings I toggle on “Remove geo location (from photos & videos that you share by link, but not by other means).” When I’m out in the field I simply take photos with my phone cam. Later, when I want to create a single iNat observation I open Google Photos, press and drag across the pix I want to upload to my observation, click on the share icon, then click on the iNaturalist icon. The pix and the location info is uploaded and the iNat observation opens up in the iNat app. I don’t select automatic upload in the iNat app. so I can first work on preparing all the observations, and then when I’m ready to post the observations and make them public I can either click on “upload,” or I can simply press, hold, and drag down on the screen to initiate the upload process. Recently I reinstalled my iNat app (as I do somewhat regularly whenever it gets buggy and I can’t fix it any other way), and I forgot to re-do my settings. I noticed the GPS coordinates were not automatically uploading to the iNat app. I was able to open Google Photos, open one photo that was used in the observation, click on the 3 dots in the upper right of the pic to open up a menu, click on “i” (info), see the little map with the location pin, click on that map to open it in Google Maps, and then copy & paste the GPS coordinates from Google Maps into the “Location notes” in the iNat app, and thus, manually add them that way after-the-fact. That was a bit of a pain, but not as time consuming as it took you to read that process, especially after I did it a few times. Better to have all your settings correct before you make your observations so that’s all automated!

Currently I’m using my Garmin Fenix 3 watch for tracking. I use it as a fitness tracker and I’m also able to export the .gpx tracks if I want to geotag my photos.
It’s really easy with Geosetter 3 for Windows. I download the .gpx tracks into the same folder as the pictures. That way I just open the folder in Geosetter, tick the .gpx track, select all pictures and press CTRL+G to automatically geotag all pictures.
It’s a really simple process!
You need to be careful if you’re travelling and pass different timezones.

In the past I’ve used a dedicated cheap GPS logger (i-Blue 747A+) to track my travels and geotag my pictures. The battery lasted much longer than my Fenix watch, but I can easily get through a day with it when I reduce the GPS logging rate.

Currently I’m using my Garmin Fenix 3 watch for tracking. I use it as a fitness tracker and I’m also able to export the .gpx tracks if I want to geotag my photos.
It’s really easy with Geosetter 3 for Windows. I download the .gpx tracks into the same folder as the pictures. That way I just open the folder in Geosetter, tick the .gpx track, select all pictures and press CTRL+G to automatically geotag all pictures.
It’s a really simple process!
You need to be careful if you’re travelling and pass different timezones.

In the past I’ve used a dedicated cheap GPS logger (i-Blue 747A+) to track my travels and geotag my pictures. The battery lasted much longer than my Fenix watch, but I can easily get through a day with it when I reduce the GPS logging rate

urate and doesn’t jump around the way the default location services do. I then sync the tracklogs with the photos in lightroom, which is just a couple clicks.

I also just discovered that the nice old Garmin Etrex vista H I have that I had stopped using since Garmin’s windows 10 software has stopped supporting it (I can’t believe they would do that for dozens of models that are still out there in the world working perfectly) will cooperate with the free software EasyGPS to download GPX files to do the same tagging process, so I’ll probably go back to using that instead of my phone for long days. If anyone has an old GPS sitting around that isn’t officially supported on their current OS, EasyGPS can probably make it usable again.

locations to make an “occupied” polygon. This allows me to map multiple species in a single survey even when they partially overlap.

I have several small gps loggers but my favorite is the Holux M-241. It has a display that shows the time to seconds so I can sync my photo times very accurately by taking a photo of the clock. I’ve been keeping track and my Pentax K-x clock drifts about 30 seconds per month while the K-70 clock drifts by about 10 seconds per month. The Holux will also run for about 36 hours on a single rechargeable AA battery, my phone can’t even come close in remote areas where it’s struggling to find a signal.

I use Picmeta Photo Tracker to geotag after I’ve adjusted the photo times in Picasa. Picasa has a nice feature that lets you adjust the time of a group of photos by setting the actual time of the first photo and it calculates the difference and applies it to the rest. If the first photo is the picture of the clock on my gps it’s very easy to get it right.

GPS Essentials works well enough for me.

Nice video! That’s very similar to my workflow. I use the free Cyclemeter 2 app on my iPhone to track where I go and I geotag my DSLR photos using the free Darktable 4. It’s a great combo.

If I realise later that my DSLR clock had drifted off time by many seconds, I can also add that offset to Darktable when I geotag to get the exact locations. It also lets me specify the timezone when I’m geotagging, which is helpful if I’ve come back from a trip overseas and have a big bunch of photos waiting to geotag.

= = = =
OK, here is how you can see your options: In the Community ID box, click "compare", then set the Place to "South America", and for Taxon, type in Calycopis, you'll get: these 4 matches in that genus: C.origo, C.isobeon, C.caulonia, C.cerata. But then try "Lycaenidae" (the larger family), and you'll get Ministrymon azia, and Strymon melinus added to that list of 6 very similar species. You may need a more local expert.
https://www.inaturalist.org/observations/identotron?observation_id=9344400&taxon=47157#establishment_means=&order=&place=97389&taxon=47923
= ==

Posted on April 16, 2019 20:29 by ahospers ahospers | 0 comments | Leave a comment