bash – AgileAdam.com

Grouping Focus Bracketed Photos from an OM-1 Mark 2

/ July 12, 2025

The Problem

I have a folder of image (JPG and RAW) files of some focus bracketed shots from my OM-1 Mark 2. I need to find a way to group them… maybe there is exif data that’d help? Or maybe I can use bash to group by timestamp to find all shots within 1 second of the previous shot… if the gap is more than 1 second then it starts a new “series”.

That was my prompt for Claude.

Claude suggested I try this first, to see what’s available:

# Install exiftool if you don't have it
# brew install exiftool  # macOS
# sudo apt install libimage-exiftool-perl  # Ubuntu/Debian

# Check what EXIF data is available
exiftool P7120919.JPG | grep -i bracket
exiftool P7120919.JPG | grep -i focus
exiftool P7120919.JPG | grep -i sequence

# Install exiftool if you don't have it

# brew install exiftool # macOS

# sudo apt install libimage-exiftool-perl # Ubuntu/Debian

# Check what EXIF data is available

exiftool P7120919.JPG | grep -i bracket

exiftool P7120919.JPG | grep -i focus

exiftool P7120919.JPG | grep -i sequence

I ran the commands and replied:

Ooh good news. There appears to be some metadata in both JPG and ORF files.
Focus Bracket Step Size : 5
White Balance Bracket : 0 0
Drive Mode : Focus Bracketing, Shot 7; Electronic shutter

Claude then gave me a bash script. I made a few tweaks, re-prompted, made some tweaks, then it was done.

It turns this flat folder of files:

P7120857.JPG
P7120857.ORF
P7120858.JPG
P7120858.ORF
P7120859.JPG
P7120859.ORF
P7120860.JPG
P7120860.ORF
P7120861.JPG
P7120861.ORF
P7120863.JPG
P7120863.ORF
P7120864.JPG
P7120864.ORF
P7120865.JPG
P7120865.ORF

P7120857.JPG

P7120857.ORF

P7120858.JPG

P7120858.ORF

P7120859.JPG

P7120859.ORF

P7120860.JPG

P7120860.ORF

P7120861.JPG

P7120861.ORF

P7120863.JPG

P7120863.ORF

P7120864.JPG

P7120864.ORF

P7120865.JPG

P7120865.ORF

Into this:

├── focus_bracket_series_1
│   ├── P7120857.JPG
│   ├── P7120857.ORF
│   ├── P7120858.JPG
│   ├── P7120858.ORF
│   ├── P7120859.JPG
│   ├── P7120859.ORF
│   ├── P7120860.JPG
│   ├── P7120860.ORF
│   ├── P7120861.JPG
│   └── P7120861.ORF
├── focus_bracket_series_2
│   ├── P7120863.JPG
│   ├── P7120863.ORF
│   ├── P7120864.JPG
│   ├── P7120864.ORF
│   ├── P7120865.JPG
│   ├── P7120865.ORF

├── focus_bracket_series_1

│ ├── P7120857.JPG

│ ├── P7120857.ORF

│ ├── P7120858.JPG

│ ├── P7120858.ORF

│ ├── P7120859.JPG

│ ├── P7120859.ORF

│ ├── P7120860.JPG

│ ├── P7120860.ORF

│ ├── P7120861.JPG

│ └── P7120861.ORF

├── focus_bracket_series_2

│ ├── P7120863.JPG

│ ├── P7120863.ORF

│ ├── P7120864.JPG

│ ├── P7120864.ORF

│ ├── P7120865.JPG

│ ├── P7120865.ORF

Implementing

Write this script somewhere (e.g., ~/scripts/om1-focus-bracket-grouper.sh )
Make it executable ( chmod +x om1-focus-bracket-grouper.sh )
Go into the flat folder (e.g., cd ~/Pictures/MacroShots )
Run it ( ~/scripts/om1-focus-bracket-grouper.sh )
It will show you what the results will be and then prompt you to hit Y to move the files or N to abort.

#!/bin/bash

# Analyze focus bracket sequences (dry run)
analyze_focus_brackets() {
    echo "=== Focus Bracket Analysis ==="
    echo
    
    local series_num=1
    local prev_shot_num=999
    local series_files=()
    
    # Sort by filename (chronological order) instead of modification time
    for file in $(ls P*.ORF P*.JPG 2>/dev/null | sort); do
        [[ -f "$file" ]] || continue
        
        # Get both drive mode and timestamp for verification
        drive_mode=$(exiftool -DriveMode -T "$file" 2>/dev/null)
        timestamp=$(exiftool -DateTimeOriginal -d "%Y-%m-%d %H:%M:%S" -T "$file" 2>/dev/null)
        
        echo "Processing: $file - Drive Mode: '$drive_mode'"
        
        if [[ "$drive_mode" =~ "Focus Bracketing, Shot "([0-9]+) ]]; then
            shot_num="${BASH_REMATCH[1]}"
            echo "  Found shot number: $shot_num"
            
            # If shot number reset (went backwards), we found a new series
            if [[ $shot_num -lt $prev_shot_num ]]; then
                if [[ ${#series_files[@]} -gt 0 ]]; then
                    echo "Series $((series_num-1)): ${#series_files[@]} shots"
                    printf "  %s\n" "${series_files[@]}"
                    echo
                fi
                
                series_files=()
                echo "--- Starting Series $series_num ---"
                ((series_num++))
            fi
            
            series_files+=("$file (Shot $shot_num, $timestamp)")
            prev_shot_num=$shot_num
        else
            echo "  Not a focus bracket shot, skipping"
        fi
    done
    
    # Print final series
    if [[ ${#series_files[@]} -gt 0 ]]; then
        echo "Series $((series_num-1)): ${#series_files[@]} shots"
        printf "  %s\n" "${series_files[@]}"
    fi
    
    echo
    echo "Total series found: $((series_num-1))"
}

# Function to actually move files after confirmation
move_focus_brackets() {
    echo "Moving files to series folders..."
    
    local series_num=1
    local prev_shot_num=999
    
    for file in $(ls P*.ORF P*.JPG 2>/dev/null | sort); do
        [[ -f "$file" ]] || continue
        
        drive_mode=$(exiftool -DriveMode -T "$file" 2>/dev/null)
        
        if [[ "$drive_mode" =~ "Focus Bracketing, Shot "([0-9]+) ]]; then
            shot_num="${BASH_REMATCH[1]}"
            
            if [[ $shot_num -lt $prev_shot_num ]]; then
                ((series_num++))
            fi
            
            series_dir="focus_bracket_series_$((series_num-1))"
            mkdir -p "$series_dir"
            
            echo "Moving $file to $series_dir"
            mv "$file" "$series_dir/"
            
            prev_shot_num=$shot_num
        fi
    done
}

# Run analysis first
analyze_focus_brackets

echo
read -p "Proceed with moving files? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
    move_focus_brackets
    echo "Done!"
else
    echo "Aborted. Files not moved."
fi

#!/bin/bash

# Analyze focus bracket sequences (dry run)

analyze_focus_brackets() {

echo "=== Focus Bracket Analysis ==="

echo

local series_num=1

local prev_shot_num=999

local series_files=()

# Sort by filename (chronological order) instead of modification time

for file in $(ls P*.ORF P*.JPG 2>/dev/null | sort); do

[[ -f "$file" ]] || continue

# Get both drive mode and timestamp for verification

drive_mode=$(exiftool -DriveMode -T "$file" 2>/dev/null)

timestamp=$(exiftool -DateTimeOriginal -d "%Y-%m-%d %H:%M:%S" -T "$file" 2>/dev/null)

echo "Processing: $file - Drive Mode: '$drive_mode'"

if [[ "$drive_mode" =~ "Focus Bracketing, Shot "([0-9]+) ]]; then

shot_num="${BASH_REMATCH[1]}"

echo " Found shot number: $shot_num"

# If shot number reset (went backwards), we found a new series

if [[ $shot_num -lt $prev_shot_num ]]; then

if [[ ${#series_files[@]} -gt 0 ]]; then

echo "Series $((series_num-1)): ${#series_files[@]} shots"

printf " %s\n" "${series_files[@]}"

echo

series_files=()

echo "--- Starting Series $series_num ---"

((series_num++))

series_files+=("$file (Shot $shot_num, $timestamp)")

prev_shot_num=$shot_num

else

echo " Not a focus bracket shot, skipping"

done

# Print final series

if [[ ${#series_files[@]} -gt 0 ]]; then

echo "Series $((series_num-1)): ${#series_files[@]} shots"

printf " %s\n" "${series_files[@]}"

echo

echo "Total series found: $((series_num-1))"

}

# Function to actually move files after confirmation

move_focus_brackets() {

echo "Moving files to series folders..."

local series_num=1

local prev_shot_num=999

for file in $(ls P*.ORF P*.JPG 2>/dev/null | sort); do

[[ -f "$file" ]] || continue

drive_mode=$(exiftool -DriveMode -T "$file" 2>/dev/null)

if [[ "$drive_mode" =~ "Focus Bracketing, Shot "([0-9]+) ]]; then

shot_num="${BASH_REMATCH[1]}"

if [[ $shot_num -lt $prev_shot_num ]]; then

((series_num++))

series_dir="focus_bracket_series_$((series_num-1))"

mkdir -p "$series_dir"

echo "Moving $file to $series_dir"

mv "$file" "$series_dir/"

prev_shot_num=$shot_num

done

}

# Run analysis first

analyze_focus_brackets

echo

read -p "Proceed with moving files? (y/N): " -n 1 -r

echo

if [[ $REPLY =~ ^[Yy]$ ]]; then

move_focus_brackets

echo "Done!"

else

echo "Aborted. Files not moved."

Development, Tech Tips

Handling Daylight Saving Time in Cron Jobs

/ March 16, 2023

Our business is located in Maine (same timezone as New York; we set clocks back one hour in the Fall and ahead one hour in the Spring)
Server A uses ET (automatically adjusts to UTC-4 in the summer, and UTC-5 in the winter)
Server B uses UTC (not affected by Daylight Saving Time) – we cannot control the timezone of this server

We want to ensure both servers are always referencing, figuratively, the same clock on the wall in Maine.

Here’s what we can do to make sure the timing of cron jobs on Server B (UTC) match the changing times of Server A (ET).

# Every day at 11:55pm EST (due to conditional, commands only run during Standard Time (fall/winter)
55 4 * * * [ `TZ=America/New_York date +\%Z` = EST ] && php artisan scrub-db >/dev/null 2>&1

# Every day at 11:55pm EDT (due to conditional, commands only run during Daylight Saving Time (spring/summer)
55 3 * * * [ `TZ=America/New_York date +\%Z` = EDT ] && php artisan scrub-db >/dev/null 2>&1

# Every day at 11:55pm EST (due to conditional, commands only run during Standard Time (fall/winter)

55 4 * * * [ `TZ=America/New_York date +\%Z` = EST ] && php artisan scrub-db >/dev/null 2>&1

# Every day at 11:55pm EDT (due to conditional, commands only run during Daylight Saving Time (spring/summer)

55 3 * * * [ `TZ=America/New_York date +\%Z` = EDT ] && php artisan scrub-db >/dev/null 2>&1

Only one of the php artisan scrub-db commands will execute, depending on the time of year.

Development, Tech Tips

Automatically Put Files Into a YYYY/MM Directory Structure

/ June 6, 2018

#!/usr/bin/env bash

## This script will find .sql and .gz files in the current
## directory and move them into YYYY/MM directories (created
## automatically).

BASE_DIR=$(pwd)

## Find all .sql and .gz files in the current directory
find "$BASE_DIR" -maxdepth 1 -type f -name "*.sql" -o -name "*.gz" |
 while IFS= read -r file; do

    ## Get the file's modification year
    year=$(date -d "$(stat -c %y $file)" +%Y)

    ## Get the file's modification month
    month=$(date -d "$(stat -c %y $file)" +%m)

    ## Create the YYYY/MM directories if they don't exist. The -p flag
    ## makes 'mkdir' create the parent directories as needed so
    ## you don't need to create $year explicitly.
    [[ ! -d "$BASE_DIR/$year/$month" ]] && mkdir -p "$BASE_DIR/$year/$month";

    ## Move the file
    mv "$file" "$BASE_DIR/$year/$month"
    echo "$file" "$BASE_DIR/$year/$month"
done

#!/usr/bin/env bash

## This script will find .sql and .gz files in the current

## directory and move them into YYYY/MM directories (created

## automatically).

BASE_DIR=$(pwd)

## Find all .sql and .gz files in the current directory

find "$BASE_DIR" -maxdepth 1 -type f -name "*.sql" -o -name "*.gz" |

while IFS= read -r file; do

## Get the file's modification year

year=$(date -d "$(stat -c %y $file)" +%Y)

## Get the file's modification month

month=$(date -d "$(stat -c %y $file)" +%m)

## Create the YYYY/MM directories if they don't exist. The -p flag

## makes 'mkdir' create the parent directories as needed so

## you don't need to create $year explicitly.

[[ ! -d "$BASE_DIR/$year/$month" ]] && mkdir -p "$BASE_DIR/$year/$month";

## Move the file

mv "$file" "$BASE_DIR/$year/$month"

echo "$file" "$BASE_DIR/$year/$month"

done

Development, Tech Tips

Find and Open (in vim) Multiple Files

/ May 9, 2018

This is a quick set of examples for finding and opening multiple files in Vim.

# Open all found files in vimdiff
find . -name '.lando.yml' -exec vimdiff {} +

# Open all found files, one-by-one
find . -name '.lando.yml' -exec vim {} \;

# Open all found files in tabs
find . -name '.lando.yml' -exec vim -p {} +

# Open all found files in vertical splits
find . -name '.lando.yml' -exec vim -O {} +

# Open all found files in horizontal splits
find . -name '.lando.yml' -exec vim -o {} +

# Open all found files in vimdiff

find . -name '.lando.yml' -exec vimdiff {} +

# Open all found files, one-by-one

find . -name '.lando.yml' -exec vim {} \;

# Open all found files in tabs

find . -name '.lando.yml' -exec vim -p {} +

# Open all found files in vertical splits

find . -name '.lando.yml' -exec vim -O {} +

# Open all found files in horizontal splits

find . -name '.lando.yml' -exec vim -o {} +

Development

Monitoring a Drupal 8 Migration Executed with Drush

/ September 7, 2017

Update 2 is proving to be the most useful method.

Update 1

Somehow I missed the documentation regarding the --feedback argument for the drush mi command. It’s similar to the methods below in that it helps you see that the migration is working, though you don’t see the totals. You can ask for feedback in seconds or items.

Update 2:

Another method that is a bit dirty but very useful is to add a single line to core/modules/migrate/src/MigrateExecutable.php.

Adding the highlighted line to \Drupal\migrate\MigrateExecutable::import will let you see exactly where the migration is (which row is being processed).

If this is something you want permanently, consider posting an issue on d.o to request more verbose logging. This is my hackey solution for my immediate needs.

$destination = $this->migration->getDestinationPlugin();
while ($source->valid()) {
  $row = $source->current();
  $this->sourceIdValues = $row->getSourceIdValues();
  error_log(date('c') . ' Processing row ' . $this->counter . ' of ' . $this->migration->id() . ': ' . json_encode($this->sourceIdValues));

  try {
    $this->processRow($row);
    $save = TRUE;

$destination = $this->migration->getDestinationPlugin();

while ($source->valid()) {

$row = $source->current();

$this->sourceIdValues = $row->getSourceIdValues();

error_log(date('c') . ' Processing row ' . $this->counter . ' of ' . $this->migration->id() . ': ' . json_encode($this->sourceIdValues));

try {

$this->processRow($row);

$save = TRUE;

The output (in your logs, and in the drush output) will be something like this (institution_node is my migration, inst_guid is the only id):

As of Thursday, August 16, 2018 the counter() method is removed. So, you have to drop a counter variable in place like this:

This method of monitoring execution is surprisingly effective as it’s fast and you don’t have to lift a finger for it to happen each time you run a migration.

Original Post

This isn’t exactly an ideal solution, but it will do in a pinch. I’m working on a migration of more than 46,000 nodes (in a single migration among many). I needed a way to monitor the progress but I wanted to do it as quickly as possible (no coding). Here’s what I came up with, which I run in a new iTerm window.

All we’re doing is asking how many records are in the “member_node” migration map every 5 seconds. If we started at 6350 items we know that by the end of 20 seconds we have created 35 records.

We could also query the target content type itself:

Here we can analyze the data the same way to see how many records the migration is creating.

I recognize that this is pretty crude, but it’s effective; you are able to see that it’s working, instead of just staring at an unchanging prompt and watching your CPU jump.

Development

Recursively Finding and Operating on the Largest n Number of Files of a Particular Type

/ February 3, 2016

If you want to reverse-engineer this, you can work from left to right, adding one command at a time to see what each step does to the output.

Shell

du -ah . | grep -v "/$" | sort -rh | grep "jpg" | head -25 | cut -f 2 | while read -r line ; do ls -alh "$line"; done

1

du -ah . | grep -v "/$" | sort -rh | grep "jpg" | head -25 | cut -f 2 | while read -r line ; do ls -alh "$line"; done

This example just lists the files we found, which is pointless given that’s what we already had before introducing the cut command.

How about a practical example?

Here’s how you can resize the 25 largest jpg files using mogrify (part of ImageMagick) to reduce the quality to 80:

Shell

du -ah . | grep -v "/$" | sort -rh | grep "jpg" | head -25 | cut -f 2 | while read -r line ; do mogrify -quality 80 "$line"; done;

1

du -ah . | grep -v "/$" | sort -rh | grep "jpg" | head -25 | cut -f 2 | while read -r line ; do mogrify -quality 80 "$line"; done;

What if you wanted to only grab files that are larger than a specific size? One way to do it is with the find command. For example, find all files greater than 8M:

Shell

find . -type f -iname "*.jpg" -size +8M -exec du -ah {} \; | sort -rh | etc...

1

find . -type f -iname "*.jpg" -size +8M -exec du -ah {} \; | sort -rh | etc...

read more
Tech Tips

Delete Last Command from Bash History

/ December 13, 2014

If you’re like me, on occasion you accidentally (or sometimes purposefully) include a password in a command. Until recently I would execute history , find the offending command’s number, then do a history -d NUMBER . It is kind of a pain.

The following deletes the last item in your bash history:

history -d $((HISTCMD-2)) && history -d $((HISTCMD-1))

1

history -d $((HISTCMD-2)) && history -d $((HISTCMD-1))

You can create an alias for this, type it manually (umm, no thanks), or, as is the case for me, create a Keyboard Maestro macro for it. Here’s what my macro looks like:

read more
Tech Tips

Customizable Date variables in Keyboard Maestro

/ June 5, 2014

UPDATE: Though the solution below works well, I do recommend following the first commenter’s advice and using the ICUDateTime text tokens instead, which allow you to use any ICU date format, without having to invoke a shell script.

Sometimes you need a date and/or time variable in your Keyboard Maestro macros. ~~The easiest~~ One way I’ve found to do this is via an “Execute Shell Script” action. You’ll just use the date command and format as desired.

read more
Tech Tips

Determining Your Most Used Commands in Terminal

/ April 7, 2014

I’m always looking to automate things using Alfred, Keyboard Maestro, Text Expander, and Python. I was curious which terminal commands I use most often, so I did some experimenting. Basically I wanted to know how many times I’ve executed each unique command ( ssh myserverx or ssh myservery, not just ssh). I started by piping the output of history to sort (to group), then to uniq (to count), then back to sort (to sort by the number of occurrences).

history | sort | uniq -c | sort -n

1

history | sort | uniq -c | sort -n

Unfortunately, the result contained the same number of lines as the original history output. I figured this was because each line had a unique integer prefix (the line number in the history output). As it turns out, ~/.bash_history (where history gets its records) doesn’t contain line numbers. I simply changed the source and it worked like a charm:

cat ~/.bash_history | sort | uniq -c | sort -n

1

cat ~/.bash_history | sort | uniq -c | sort -n

read more

Tech Tips

“755”-style permissions with ‘ls’

/ February 5, 2011

After a quick Google search for “ls permissions octal” I found a very handy alias to put in my .bashrc. Not only is it handy to see the OCTAL value of the permissions for each file/dir, it will undoubtedly help you more quickly recognize/interpret the normal ls -al output.

alias lso="ls -alG | awk '{k=0;for(i=0;i<=8;i++)k+=((substr(\$1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf(\" %0o \",k);print}'"

1	alias lso="ls -alG \| awk '{k=0;for(i=0;i<=8;i++)k+=((substr(\$1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf(\" %0o \",k);print}'"

Example usage:

mycomp:~ adam$ lso
755 drwxr-xr-x 15 adam staff 510 Dec 6 02:47 .vim
644 -rw-r--r-- 1 adam staff 1136 Dec 18 16:55 .vim_mru_files
600 -rw------- 1 adam staff 13665 Dec 18 16:56 .viminfo

mycomp:~ adam$ lso

755 drwxr-xr-x 15 adam staff 510 Dec 6 02:47 .vim

644 -rw-r--r-- 1 adam staff 1136 Dec 18 16:55 .vim_mru_files

600 -rw------- 1 adam staff 13665 Dec 18 16:56 .viminfo

You can also type (or paste) this slightly different version into terminal:

ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf("%0o ",k);print}'

1	ls -l \| awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf("%0o ",k);print}'