Cloud monitor / All sky camera on a budget

Affiliation
American Association of Variable Star Observers (AAVSO)
Thu, 06/11/2020 - 23:00

Either for observing meteor showers, or just for checking incoming clouds, all-sky cameras are a useful addition for many observatories.

Some of us who bought a ZWO ASI CMOS camera in the "planetary" or guiding market segment like the ASI178 , ASI 120, ASI 290 etc actually got a simple fish eye lens for free because ZWO just throws in one of these as an accesory when you buy those cameras. These lenses, while simple, actually have a CS mount thread and are screwed into a T2 adapter ring to fit the ZWO ASI cameras.

If you always wanted to have a cheap all-sky camera. you could actually make use of these lenses with an inexpensive camera module: The Raspberry Pi Foundation has recently released a rather interesting camera module for the Raspberry Pi credit-card size computers, the "High Quality camera module", for just around 50 US$ (!). Together with a Raspberry Pi, you could then build a simple all-sky camera system for well under 100 IS$ .

Unlike the previous Raspberry Pi camera modules which were basesd on low-end smart phone sensors, this one has a rather decent sensor, the Sony IMX477 , and it comes packaged with a CS mount so you can easily adapt almost any lenses, even telescopes. There is also a way to remove the IR-cut filter to make it IR sensitive.

This is a color sensor with really tiny pixels and shallow full well capacity, so definitely not optimal, if usable at all, for photometry by any means, but for a simple all-sky camera, it seems adequate, and its low-light performance is not bad at all https://photo.stackexchange.com/questions/116765/new-pi-camera-any-good…

So I ordered and finally got one, and will try to turn it into a reasonable all-sky camera and maybe even a moon & planetary video capture camera (the sensor was actually designed with camcorders in mind).

But I guess the camera/sensor part is pobably not the most difficult part in building an all-sky camera... environmental protection (rain/dew/frost/sun/UV) comes to mind... 

It would be great if people could share their experiences here.

CS

HB

Affiliation
American Association of Variable Star Observers (AAVSO)
Cloud monitor / All sky camera on a budget

I'd seen ads for this camera, and it looked like it might be useful. Thanks for ordering the camera. If it looks good, I've got two RPi's and would love to learn more about its performance. I'm still not up to snuff on using Linux, so might lean on you for more info on installing and configuring it, Thanks for posting.
I'd also be curious how it performs as an astrocamera, not just an all-sky camera. 

--Richard.

Affiliation
American Association of Variable Star Observers (AAVSO)
Unreliable cheap cameras

I have a keen interest in allsky cameras as well. I have used two different ones, and each had their problems.

When hosting the BSM here in Hawaii, we used the Moonglow ASC. The main issues it had was: 1. A very low resolution sensor, the large pixels were very obvious, and resulted in poor rendition of the fainter stars. 2. It would slowly acquire more and more "hot pixels" which would mar the image and give false positives of transient objects. Though the software has the ability to blank these out, it required constant attention. 3. Eventually, some part of the device or interface would fail for unknown reasons. The camera had to be sent back for service, and the really weak part was the usb dongle. It would frequently fail or drop connections. Overall, while it did provide a usable image, the poor reliability, and not so cheap cost, gives it a "fair-poor" rating. And now, it is no longer supported by the company, so is practically unavailable and unusable anymore :(

Second one was the ZWO ASI120 with its free "fish eye" lens. This also had problems. The sensor had good resolution, gave a much more detailed image than the Moonglow, but somewhat less sensitivity. Maybe one magnitude less, and required really long exposures at high gain to see the milky way faintly. Pretty high noise levels at these setting too. The problems: 1. Constant system failures of one kind or another. I tried several units, the company sent me replacements each time. Most often, the image would suddenly acquire a "raster pattern" over half the image! No solution to this problem was found, it would occur randomly. Sometimes it would drop the usb connections, show the dreaded "hourglass", again randomly. 2. It didn't last long, running 24/7 exposed to the outdoor temperature and humidity variations. Even though protected from the elements directly, the harsh outdoor environment quickly killed it. After a month or two like this, it failed completely.

So, in conclusion, these two "cheap" cameras experienced numerous hardware issues requiring a lot of debugging time and interface with customer services. The ZWO ASI did not last very long running continuously in an outdoor-like environmental condition, likely it is designed only for occasional use, not 24/7.

Affiliation
American Association of Variable Star Observers (AAVSO)
Useful to know. I've been

Useful to know. I've been using the ZWO120 and its included 120-degree "fisheye" lens, but it's only been turned on at night through the last autumn and winter, and has never been exposed to summertime temperatures yet. The Milky Way is clearly visible in 5-second exposures. The odd imaging problems you've experienced suggest a USB3 cable problem.

Does anyone else have experience with fisheye all-sky cameras?

Richard

Affiliation
American Association of Variable Star Observers (AAVSO)
Out of curiosity I briefly

Out of curiosity I briefly tried my ZWO ASI 178mm with the included lens once, and I think it was quite OK. But after that I permanently married that camera to my telescope, and indeed I would have hesitated to use the 400 EUR camera in an all-sky-setup 24/7. I agree with Michael that they are just not inteded for that use case. With the new Raspberry Pi camera tho, even if the camera should fail because of ...whatever...water damage, theft, vandalism by wild animals...you name it, the loss would be minimal and easy to replace.

What's nice about the Raspberry Pi community is that most problems you encounter have already been worked out by others, e.g. some people have figured out how to extract raw images (by extending the dcraw code) and make long exposures of up to ~ 200sec with short read-out time.

But of course, the next couple of days here will be cloudy/rainy :-( .... stay tuned. I'm already quite confident this will outperform my current cloud monitor which is based on one of the older Raspberry Pi camera modules and one of those super-cheap add-on lenses you can buy for smart-phones.... the quality is very poor but at least you can spot clouds from the comfort of an indoor desk, better than with the naked eyes.

CS

HB

Affiliation
American Association of Variable Star Observers (AAVSO)
Well, this is not the place &

Well, this is not the place & time of year to test an all.sky cam setup because it's actually not getting really dark here anymore....

Anyway, this is a link to a 30 second, out of the camera JPG image with the new Raspberry PI HQ camera and the ZWO ASI lens that came with my ZWO ASI 178mm, at 01:00 local daylight saving time. It doesn't get darker than this :-(.

The focus seems OKish and while postprocessing could be used to bring out the stars more clearly (blur filter), if this  is to be used as a cloud monitor this is already pretty ok the way it is. The quality degrades near the edges/horizon but that is to be expected for such a simple/inexpensive lens.

http://bikeman.selfhost.eu/astro/IMX477-tests/focus_test_30sec.jpg

I guess the cat-net is a nice test pattern :-)

Longer exposures lead to star trails and won't do any good at the native resolution, binning is a different matter.

CS

HB

Affiliation
American Association of Variable Star Observers (AAVSO)
Nice image, looks like

Nice image, looks like moonlit sky!

I'm located at 58 N and here is the perfect season for flat frames... Sun sets ~8.2 degrees below the horizon at the local midnight. High-quality photometry in UBV is practically impossible for any stars that would require exposure times longer than few seconds. Fortunately, high(er) resolution spectroscopy can still be done for ~2.5 hours per twilight. And bright (nake eye) stars can be measured with modern CMOS cameras, too.

We use here an Oculus Allsky camera (USB connection + power for heating) and it works reasonably well. It's focusing can be a bit challenging but so far over 1.5 years it has worked very well. It's not very cheap but definitely not the most expensive one.

Best wishes,
Tõnis

Affiliation
American Association of Variable Star Observers (AAVSO)
This is a very nice image!

This is a very nice image! Far better than the old Moonglow ASC. So, I would like to get one and use it, but I have no experinece with the Raspberry pi (though this would be a good opportunty to learn LOL).

In the meantime, do you think there would be a way to capture images from this camera using some kind of Windows software?

Mike

 

Affiliation
American Association of Variable Star Observers (AAVSO)
I don't think Windows is an

I don't think Windows is an option. The camera module comes with a CSI ribbon-cable interface, even if you find a board that somehow bridges this to USB for Windows PCs, you would still need drivers & firmware etc .... certainly easier to just buy a Raspberry Pi and research the few steps it takes to turn it into an appliance that streams images or puts them on a Web-Server running on the Pi or ftp them to a Windows box or stores the images on a shared Windows drive in the network. 

Tönis mentioned heating. The nice thing is that the camera module would be co-located with the Raspberry Pi. It's easy to connect a temperature and humidity sensor to the RasPi and if the need arises, it could automatically start some compute-intensive tasks on, say, 3 of the 4 CPU cores, which would generate a few extra Watts of heat :-), which might already be enough depending on your environmental conditions. Newer models also have an option for a cheap  "Power over Ethernet" extension module for easy wiring. 

CS

HB

Affiliation
American Association of Variable Star Observers (AAVSO)
Yes, I definitely want to try

Yes, I definitely want to try this cmera and Pi! However I have essentially ZERO kowledge of this little computer. Could you recommend some sources of info (preferably free/internet) to help me get started?

1. An "executive summary" overview of the system and theory of operation.

2. A beginners guide or tutorial to get started with it.

3. Some more detailed guides on whether and how you can access the system by your own custom C/C++ program?

Thanks, Mike.

Affiliation
American Association of Variable Star Observers (AAVSO)
Most of the beginner's

Most of the beginner's documentation by the Raspberry Pi Foundation is targeted for kids or teenagers, probably because the Raspberry Pi Foundation is a non-profit charity and to maintain this status it has to embrace a charitable cause like educating the young in computer skills, rather than appealing to grown-up  "Makers".

But they are well written, so if you are not repelled too much by the youth-friendly page design & language, this here is actually a very nice beginner's guide

https://magpi.raspberrypi.org/books/beginners-guide-3rd-ed/pdf

You can skip chapter 4 which is really just for kids.

The entire MagPi magazine, which has a for-sale paper edition but is also downloadable in PDF for free, is a treasure trove of tutorials and example projects.

A bunch of free tutorials can be found here: https://magpi.raspberrypi.org/articles/category/tutorials ranging from simple things like connecting a temperature and humidity sensor to receiving radio transmissions from the ISS, learning Artificial Intelligence programming etc etc etc

Most of these tutorials will use Python, not C/C++ as the preferred programming language. However, this is something that is also a trend in academia in general and astronomy in particular. Software packages like AstroPy  https://www.astropy.org/  are now defacto must-have tools which can be very useful for tasks like FITS file IO, celestrial coordinate operations, time standard conversions, observability calculations (sun rise/set time, Alt/Az  angles, airmass calculations)..... whatever.

Any book / online tutorial etc for Python will do, there is nothing different for the Raspberry Pi here. People familiar with C will learn Python rapidly.

But of course you can do C/C++ programming as well on the Raspberry PI, again it's a full-blown computer, running on Linux (a version of the Debian Linux distribution to be specific). It's just that most of the add-on hardware that you can buy for the Pi (environmental sensors, LCDs, TFTs, ePaper displays, motion sensors, ... also anything that can connect to an I²C bus ! ) will likely have example code provided in Python, rather than C these days.

Another source for add-on products and tutorials (text and videos) is the Adafruit store, https://www.adafruit.com/ and this is more targeted at grown up "Makers".

As for the hardware, the Raspberry Pi comes in different performance levels and form factors.

If the focus is on low power consumption and/or small form factor rather than computing performance, there is the "Raspberry Pi Zero W" https://www.raspberrypi.org/products/raspberry-pi-zero-w/ which has a single-core CPU, NO Ethernet (!), but WiFi, and a single USB port, at a rediculously small form factor. Even that one can connect to a camera, as the image processing is done in dedicated hardware in a GPU part of the CPU that is fast enough to deal with the image data. I use that one in my first portable cloud monitor (see enclosed picture, always wanted to put that in carry-on luggage an see what happens ....)

The other models have more or less the credit-card form-factor and differ in processing power and amount of RAM, and of course price tag. The "Zero W" model is about 10 US$ , other models like the "model 3B+" are 35 $ and the top notch "model 4B" with up to  8 GB of RAM (!) , USB 3 and Gigabit Ethernet  is the most "expensive" one at 75$ (for the 8GB version). For all but the most demanding tasks, the model 4B with 4GB RAM should do (55$). Make sure you get a good power supply, not just any cheap USB phone charger, and a quality SD card.

BTW, many of you will be familiar with the ZWO AsiAir and now AsIAir Pro  products https://astronomy-imaging-camera.com/product/asiair-pro. Those are driven by Raspberry Pis , so this gives you an idea what the computing power of a  Raspberry Pi can do in connection with astronomy.

I use an older Raspberry Pi Model 2A on my quick-look-for-fun solar telescope based on a Celestron First Scope (the Raspberry Pi is under the TFT touch screen).

I have another PI  running a web-browser showing the current astronomical seeing forecast because i found a TFT screen that fits snuggly into one of the cutout compartments of my IKEA whiteboard (it can also play internet radio :-) ). These are just a few examples for "astronomical" applications of the Raspberry Pi.

Warning: Once you are into it, this can get a bit addictive:-) I stopped counting how many RasPis I own, but it must be close to 20 or so, in various appliances (three in seismometers, air traffic recording stations, media servers, web servers, file servers, photo traps to spy on my cats , pulse oximeter, serial interfaces for vintage computers, receivers for NASA's GCN notices, home automation, weather satellite image receivers ....) It's great fun.

CS

HB

Affiliation
American Association of Variable Star Observers (AAVSO)
ZWO ASI120 camera

 

I have been using the ASI120 camera  with the free fish eye lens for a couple of years now as a cloud monitor. I have been very satisfied with it. It is mounted in my observatory and is subject to southwest desert temperture conditions and seems to be working great. With the very dark sky here, exposures approaching 60 seconds are sometimes needed to see clouds on moonless nights.

 

 

 

 

 

Affiliation
American Association of Variable Star Observers (AAVSO)
You must have been lucky and

You must have been lucky and got a good one! I had the same problems of usb disconnects, image failures with partial raster patterns - with 2 different ASI120 units. The manufacturer sent me a free replacement, but the same problems showed up after a short time of 24/7 operation! Plus, these problems occured at 2 very different environmental conditions. Both at our relatively benign Hawaiian place, as well as our desert location in California, with wide temperature swings. And two different laptops, runing different versions of Windows.

I just gave up wasting time trying to get this model to work for me as an all-sky. Too many variables and no definite solutions, despite a lot of time and effort wasted with the ZWO.

Mike

Affiliation
American Association of Variable Star Observers (AAVSO)
That is VERY impressive. 

That is VERY impressive. 
We need to collect the information you have sent and archive it in the I&E Observing Section.

The SAS/AAVSO meeting is on now, but soon as I get a bit of time, I'll figure out how to do that.

--Richard  

Affiliation
American Association of Variable Star Observers (AAVSO)
Hi Brian!

Hi Brian!

The Raspi HQ camera is special in that it's interface is not USB, but it connects to the Raspberry Pi via CSI, which has both pros and cons. Much progress has been made by the wizards who do  firmware and acquisition software for this thingy lately.

Yes, you can get "raw" output from the sensor, and there is even a fork of the popular dcraw s/w  that can handle those raw images. See

https://github.com/6by9/dcraw

https://github.com/6by9/raspiraw.git

https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=276047#p1675364

It requires a bit of tweaking, most likely updating your firmware manually at the time of writing, but can be done.

I put "raw" in quotes because by default, the sensor will have a "star eater" filter active (even applied to the "raw" output, that automatically "manages" bad pixels..but will mess us photometry). With newer firmware, this can be switched off to give a truely raw output:

https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=277768

Exposure length and gain can be controlled manually,  exposure length can be up to a bit less than 4 minutes,  but given the 12 bit ADC, tiny 1.55 μm  pixels and (i guess) correspondingly shallow well depth, I guess one would rather take advantage of the typically low read noise of those CMOS sensors and use stacked, shorter exposures.

I have no idea about the sensitivity, but this is a relatively  recent (Dec 2016), back side illuminated sensor, so I'm optimistic.

Cheers

HB

 

Affiliation
American Association of Variable Star Observers (AAVSO)
Some tests thru a 8" Newtonian

I finally managed to do some quick tests of the ca 50 US $  Raspberry Pi HQ camera module (Sony IMX 477 sensor)  thru my 8" Newton. As my initial intention was to test it on planetary targets, I also used a Barlow lens infront, I'll do some tests later w/o a Barlow. I ended up doing some test shots of M57.

So here is what I did:

- 38 light frames, exposure 30 seconds , "analog gain" exposure gain setting 10.

Raspistill command line:

raspistill -t 10 -md 3 -bm -ex off -ag 10 -ss 30000000 -st -r -o $filename

I deactivated the "star eater" feature: 

sudo vcdbg set imx477_dpc 0

- 40 dark frames

- no flat field correction yet....will come later

The -r option will cause raspistill to embed the raw sensor data as metadata into the jpg that it produces. That raw data was then extracted and written to DNG files (all on the Raspberry Pi) with the "PyDNG" tool found here :

https://github.com/schoolpost/PyDNG

The light and darjk frames were coadded with AstroArt, using "Green channel 2x2 binning" , with just pixel defect mitigation enabled but no other processing done.

The resulting green-channel-only stacked image is here: http://bikeman.selfhost.eu/astro/raspi_hq_test/M57_green2x2_test2_imx47…

I did some very quick photometry checks and it seems to be "plausible" to the extend that I think nothing obviously is wrong with it, so the extraction of the green channel from raw sensor data seems to have worked.  

Just for fun, I produced a "pretty picture" with DeepSkyStacker and NeatImage ... yeah well this doesn't exactly re-invent astrophotography but it's a nice 50 $/EUR/pound thingy to play with.

http://bikeman.selfhost.eu/astro/raspi_hq_test/M57_dss_test1_28x30s_imx…

A single DNG frame (30 seconds) is here: http://bikeman.selfhost.eu/astro/raspi_hq_test/2020-09-18T212156.dng

I still see this more as an inexpensive all sky camera, but it's good to know it can do decent OSC as well.

Cheers

HB

Affiliation
American Association of Variable Star Observers (AAVSO)
Hi....I acquired the source…

Hi....I acquired the source code from a genuinely notable designer in the cosmology network under the cover of a free NDA so I'm not allowed to share the code. Anyway I can say the product tallies stars above 30° to decide darkness. It functions admirably, yet my skies are intensely light dirtied which decreases the calculation's viability - at any rate until the observatory gets moved to my dull site close to Yellowstone NP. I intend to integrate the outcomes with a DIY climate station/sky temp screen I'm working to decide sky conditions.

pcba manufacturer with iso 13485 certificate

Affiliation
American Association of Variable Star Observers (AAVSO)
Raspberry Pi HQ Camera

I have been developing a Python3 based software for my newly acquired HQ camera for use as an all sky camera. I had an SBIG All Sky Monitor but lightning took that out. So what I am doing is taking out the fish-eye lens and mounted it on the HQ camera. I had temporarily mounted the HQ and a older RPi 2 into the SBIG housing and I used the acrylic dome for protection. I did some quick testing using the PRi Camera library and have since found out that the exposure limit has been significantly increased to allow for better all sky images. So I am continuing with my work on this and I find this to be a potentially wonderful setup.