1. Home
  2. Forums
  3. Variable Stars
  4. Data Mining

Data Mining Workshop: Jupyter Notebooks

YPFA
Affiliation
American Association of Variable Star Observers (AAVSO)
Thu, 03/18/2021 - 23:59

I have just been watching (on YouTube) the Data Mining Workshop from last November, run by Colin Littlefield.

I am about 1h 30m into the video and Colin is talking about Jupyter notebooks and perhaps other items which were sent to participants in advance.

I have installed the Anaconda distribution of Python and Lightkurve, but I cannot progress without the Jupiter notebook/s and anything else he may be about to reference in the remainder of the workshop.


How might I access the package of materials that underpins the workshop?

Regards: Paul

rachlin
Affiliation
American Association of Variable Star Observers (AAVSO)
Tue, 10/17/2023 - 23:45
Using Anaconda for Python Development

I teach courses in advanced data science programming (with Python) at NEU.  Anaconda is a python distribution which includes a) the python interpreter, b) development tools such as Jupyter Notebooks and Spyder, and c) an environment manager called "conda".   The "base" environment that is included with the anaconda installation has jupyter notebooks built in along with 400+ pre-installed packages.   You can also create your own project-specific environments, and then install just the packages you need.

I recommend having a look at a starting guide for learning conda:  https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html

If you need to install Jupyter Notebooks in a newly created environment, try opening a terminal (or run the "Anaconda Prompt" on Windows), activate your environment, and run the conda install command.   Here is a possible command sequence:

# Create a new anaconda environment called "astro" that uses version 3.11 of the python interpreter
conda create --name astro python=3.11

# Activate the new environment
conda activate astro

# Install any libraries (packages) that you need
conda install jupyter

# Navigate to a working directory where your code lives
cd /Users/rachlin/code/astro

# Run Jupyter Notebooks
jupyter notebook

 

clkotnik
Affiliation
American Association of Variable Star Observers (AAVSO)
Wed, 10/18/2023 - 00:13
Anaconda Experience

Because of the folks who describe the benefits of Anaconda I have tried it several times over the years.  Where I run into problems is the bit after getting started.  Maybe folks can help me understand it better.

For example, the last time I tried I ran into problems because astropy was a couple of release out of date.  I upgraded it to the current release.  After that, any update operation ran for the better part of an hour.  It just became unworkable.

What trick am I missing?

thx

Cliff 

rachlin
Affiliation
American Association of Variable Star Observers (AAVSO)
Thu, 10/19/2023 - 18:13
Dependency Checks

Hi Cliff,

Anaconda comes with a "base" environment that includes hundreds of pre-installed libraries and a specific version of the python interpreter many of which you are not likely to use.  One of the functions of anaconda is to manage environments: creating new environments with a designated version of python, while allowing you to add or update libraries in an existing environment.   Because libraries are built upon one another, conda does dependency checks.  Thus, installing or updating astropy might trigger the installation or updating of additional libraries in your environment, and the process can cascade.   (When anaconda says that it is "solving the environment" it is carrying out these dependency checks.   If you install A, you need B, C, and D.   B requires E, F, and G, etc.)

So I recommend, NOT using the base environment for software development but instead creating new environments and installing only the libraries that you need for your project.   You'll find that the dependency checks go faster because there are fewer dependencies to check!   Also, do keep in mind that conda supports installing libraries from alternate "channels" which might maintain more update libraries and provide faster response time.   For astropy, you should install from the conda-forge channel per their recommendation.  For example, after activating your development environment:

 

conda activate <your-environment-name>

You would then say:

conda install -c conda-forge astropy

Finally, if you want to install a version of anaconda that does not pre-install the base environment with hundreds of pre-installed packages, check out miniconda. https://docs.conda.io/projects/miniconda/en/latest/

Best,

John Rachlin (RJOJ)