Category: Other Tech

  • I Built Another Thing! (To extract Slack conversations for my “AI memory”)

    I Built Another Thing! (To extract Slack conversations for my “AI memory”)

    I’ve written about my AI memory system before, which gives Claude a “deeper” memory across conversations. I rely on this every day, but noticed that it still misses some context due to the fact that so many day-to-day conversations happen in Slack.

    Stuff like design discussions, technical opinions, those crucial “oh by the way” messages that reshape entire roadmaps, etc.

    But Slack is surprisingly lacking in terms of smart data export or analysis. For example, we have an internal helpdesk type of channel which I wanted to get a dump of data from to analyse; things like request count trends, documentation gaps, underserved teams, etc – but no luck, even when I requested this data through proper internal channels (i.e. IT).

    Anyway, I needed something that could grab specific conversations, preserve the context, and output clean markdown that my AI assistant could digest. So I built “SlackSnap”. 🎉

    Starting Simple-ish

    The good old “copy/paste” doesn’t really work here (no threads, messes up formatting, etc), so I didn’t start that simple.

    First I tried a javascript snippet that grabbed textContent from message elements. It kinda worked, but:

    • Slack’s DOM is a maze of virtual scrolling and dynamic loading, and the last time I pretended to be a “web developer” was in the 90s 🧓
    • Only captured visible messages (maybe 20-30 out of hundreds)
    • Lost all formatting (code blocks became walls of text)
    • No thread support
    • Usernames were just IDs like “U01234ABCD”

    So I rebuilt it as a proper Chrome extension. This gave me:

    • Background service workers for file downloads
    • Content scripts with full DOM access
    • Storage API for configuration
    • Proper permissions for Slack domains

    But the real breakthrough came when I discovered Slack loads its API token into localStorage. Instead of scraping the DOM, I could use Slack’s own API (well… *I* didn’t discover shit, the nice AI in Cursor informed me that this might be a better option 😄)

    Next: Dual Extraction Methods

    SlackSnap uses a two-pronged approach:

    Method 1: API-Based Extraction (Primary)

    // Get auth token from Slack's localStorage
    const config = JSON.parse(localStorage.getItem('localConfig_v2'));
    const token = config.teams[teamId].token;
    
    // Fetch messages via conversations.history
    const messages = await fetch('/api/conversations.history', {
      method: 'POST',
      body: new URLSearchParams({
        token: token,
        channel: channelId,
        limit: '100',
        oldest: oldestUnix
      })
    });

    The API approach is nice and simple (and understandable!) because it:

    • Gets ALL messages in the specified time window, not just visible ones
    • Includes thread replies with conversations.replies
    • Provides consistent data structure
    • Works with Slack’s pagination

    But the user IDs problem remained. Slack returns messages like:

    {
      "user": "U123ABC",
      "text": "Should we refactor the auth service?",
      "ts": "1753160757.123400"
    }

    Smart User Resolution

    Instead of fetching ALL workspace users, which the AI did initially, and *I* actually corrected (chalk 1 up for the humans!), SlackSnap:

    1. Extracts unique user IDs from messages
    2. Includes @mentions from message text
    3. Fetches ONLY those specific users
    4. Builds a lookup map for the export
    // Extract user IDs from messages and mentions
    const userIds = new Set();
    for (const msg of apiMessages) {
      if (msg.user) userIds.add(msg.user);
      // Extract @mentions like <@U123ABC>
      const mentions = msg.text.match(/<@([A-Z0-9]+)>/g);
      // ... collect those too
    }
    
    // Fetch only needed users (e.g., 15 instead of 5000)
    const userMap = await fetchSpecificUsers(Array.from(userIds), token);

    Method 2: DOM Fallback

    If API access fails (permissions, network issues), SlackSnap falls back to enhanced DOM scraping:

    // Scroll to load message history
    const scrollContainer = document.querySelector('.c-scrollbar__hider');
    
    for (let i = 0; i < 20; i++) {
      scrollContainer.scrollTop = 0;
      await new Promise(resolve => setTimeout(resolve, 1500));
    
      // Check if new messages loaded
      const currentCount = document.querySelectorAll('[role="message"]').length;
    
      // ... break if no new messages after 3 attempts
    }

    This bit never worked as well (still had issues resolving user names, was more inconsistent with the scrolling, etc) so I may just remove it entirely since I’ve found the API method actually works more reliably.

    The Output: Clean, Contextual Markdown

    SlackSnap produces markdown that preserves the conversation flow:

    # SlackSnap Export: #deathstar-review
    
    *Exported: November 28, 2024 10:45 AM*
    
    ---
    
    **D.Vader** (Today 9:15 AM):
    Team, what's this I hear about an "exhaust port" vulnerability?
    
    **Galen Erso** (Today 9:18 AM):
    Nothing to worry about; low sev vulnerability we can patch out later as a fast-follower :thumbsup: :agile:
    
    **Thread Replies:**
      - **Grand Moff T**: It's only 2 meters wide right? nobody's getting close enough to even see it! :approved:
      - **Emperor P**: Yeah, okay... its just a vent I guess, probably doesn't lead anywhere important in any case. Thanks team

    Configuration

    The options page lets you control:

    • Download directory: Organizes exports (e.g., downloads/slack-exports/)
    • Filename template: YYYYMMDD-HHmm-{channel}.md for chronological sorting
    • History window: How many days back to export (default: 7) – things get a bit funky if you download too much from a very busy channel
    • Thread inclusion: Critical for capturing full context, or to disable if you just want a high level overview
    • Timestamps: Full date/time vs. just time

    How I use the output

    The structured markdown output feeds directly into my AI context system. The way I do this isn’t to capture every single little detailed message, but to export (on a weekly basis) from a few key channels likely to contain important context, then pass all of those exports into Claude at once, asking it to write a single summary file to memory for that week, focusing on team dynamics, key decisions and technical direction, etc. 

    Then the memory system can take that Slack summary into account when I do my regular “memory updates”. So now when I start a chat in Claude Desktop, it can make use of context from meeting transcripts and documents I’ve provided, plus Slack conversations!

    For the week or so I’ve been using it I’ve noticed that it feels a little more accurate, or “connected to reality”, than it did before. YMMV.

    The Technical Stack

    Everything runs locally in your browser:

    • Manifest V3: Modern Chrome extension architecture
    • Slack’s Internal API: Already authenticated, just reuse their token
    • Chrome Downloads API: Handles subdirectories properly
    • Markdown Generation: Preserves code blocks, links, formatting

    Installation and Usage

    1. Clone from GitHub: https://github.com/dcurlewis/slacksnap
    2. Load as unpacked extension in Chrome
    3. Click extension icon on any Slack conversation
    4. Messages export to your Downloads folder

    The export captures the entire history (up to your configured limit), not just what’s on screen.

    Only tested on Chrome (but should work on Chromium based browsers, or those using the same extension architecture).

    Future Enhancements?

    • Selective date ranges: Export specific time periods
    • Multi-channel export: Batch export related channels
    • Search integration: Export search results
    • Attachment handling: Download shared files/images
    • Export formats: JSON for data analysis, PDF for sharing

    But honestly? The current version solves my immediate need so I probably won’t bother adding too many bells and whistles.

    Some Observations

    Building this revealed some interesting patterns in how we communicate:

    1. Critical decisions often happen in threads – Main messages lack context
    2. Code snippets in Slack are surprisingly common – And poorly preserved
    3. Timestamps matter more than you think – “Yesterday” is ambiguous a day later
    4. User attribution is crucial – “Someone suggested” vs. “Darth Vader suggested”

    Other observations from me, less related to this tool and more on the process of developing it; “Vibe coding” can still be infuriating, but works a lot better IMO if you provide a decent project plan at the outset. 

    I’ve seen arguments that planning time is better spent “iterating” (vibing?), but I literally spent 2 or 3 minutes prompting another AI to produce my “plan” based on my quickly thrown together requirements and limitations.

    This saved probably hours of the AI running me around in circles with “vibey” scope creep, mocked functions it thinks might be handy when you implement this amazing feature one day (that I definitely didn’t ask for), etc.

    Get It Running

    The tool is here: https://github.com/dcurlewis/slacksnap

    It’s intentionally simple – no external dependencies, no build process, just vanilla JavaScript that manipulates Slack’s own data. If you’re feeding an AI assistant with your work context, this might be the missing piece.

    Let me know if you find something like this useful, or if you have any feedback/ideas to share. 

    Cheers,
    Dave


    P.S. – Yes, I realize I’m slowly building a suite of tools to feed my AI assistant. Not sure what might be up next, yet…

  • I Built a Thing! (To Transcribe my Meetings)

    I Built a Thing! (To Transcribe my Meetings)

    The third-party transcription app I was using for my AI memory system got flagged as non-compliant at work. Zoom’s built-in transcription only really works when you’re the host. For vendor calls, external meetings, and anything where I wasn’t running the show, I needed an alternative that was free, local, and (more) compliant.

    So I built one.

    The Starting Point: Manual but Functional

    I already had OBS Studio installed (it’s on our approved software list) and knew it could record audio. My initial workflow was basic:

    1. Manually start OBS recording before a meeting
    2. Stop recording after
    3. Run OpenAI’s Whisper locally on the audio file
    4. Paste the transcript into Claude Desktop for summarization

    It worked, but had obvious problems:

    • Everything was manual
    • No speaker separation (just one wall of text)
    • Back-to-back meetings meant falling behind on processing
    • Easy to forget to start/stop recording

    The Evolution: From Manual to Automated

    First, I automated the OBS control using its websocket API. No more clicking around in the UI, just command-line control.

    Then I realised I could use OBS’s multi-track recording to solve the speaker problem to some extent:

    • Track 1: My microphone
    • Track 2: Desktop audio (everyone else)

    This works perfectly for 1-on-1 meetings, since there are only 2 of you, but for group meetings you’ll only know for sure what was said by you, and everyone else.

    Haven’t figured out a way to solve this yet, but to be honest AI summarization does a pretty good job in most cases of inferring who said what. It may just make mistakes in meetings where it’s important to know exactly who said what (e.g. assigning tasks or follow-up actions).

    FFmpeg could extract these tracks as separate files, Whisper could transcribe them independently, and a simple Python script could merge them with timestamps:

    [00:01:23] Me: What's the status on the inference service?
    [00:01:31] Others: Still blocked on GPU allocation...

    Finally, I opened Cursor (using Gemini 2.5) all my requirements, and asked it to build a proper CLI tool. The result was a bash script that orchestrated everything: OBS control, audio extraction, transcription, and transcript merging.

    The Final Tool: Simple Commands, Complete Workflow

    bash# Start recording
    ./run.sh start "Sprint Planning"
    
    # Process all recordings when you have time
    ./run.sh process
    
    # Check what's queued
    ./run.sh status
    
    # Discard a recording
    ./run.sh discard

    Key features I added during refinement:

    • Queue system: Records meeting metadata to processing_queue.csv for batch processing later (for when I have several back-to-back meetings and need to be able to process them later)
    • Automatic stop: Starting a new recording auto-stops the previous one (because I have a crap memory 😉)
    • Idempotent processing: Won’t re-process already completed steps if interrupted (e.g. if you start recording a meeting but nobody shows, or you don’t want to process it for some other reason)
    • Configurable Whisper models: Trade speed for accuracy based on your needs (I haven’t played with this much, so have only tried the base and small models, which worked well, but there is a turbo model too which looks interesting)

    The Technical Stack

    Everything runs locally:

    • OBS Studio: Multi-track audio recording (already approved in my use)
    • FFmpeg: Extract multiple audio tracks from MKV files
    • Whisper: Local transcription (base model by default, configurable)
    • Python: Controls OBS via websocket, merges SRT files
    • Bash: Orchestrates the workflow

    Why This Works for Me

    1. Fully local: No data leaves your machine
    2. Uses approved tools: in my case at least
    3. Handles real workflows: Queue system for back-to-back meetings
    4. Good enough quality: Whisper’s base model is sufficient for most meetings
    5. Searchable output: Timestamped, speaker-separated transcripts

    The transcripts feed directly into my AI assistant for summarization, action item extraction, and long-term context building. No manual notes, no missed decisions, no compliance issues.

    Get It Running

    The tool is here: https://github.com/dcurlewis/obs-transcriber along with more detailed instructions.

    Setup is straightforward:

    1. Install OBS Studio and enable websocket server
    2. Install FFmpeg and Python dependencies
    3. Configure OBS for multi-track recording
    4. Run the commands

    It’s not as polished as commercial services, but it solves my specific problem of local meeting transcription. Improvements I’m already thinking about are:

    • auto-deleting the recordings once they’re processed
      • cuts down disk space bloat, but importantly reduces the amount of potentially sensitive data lying around
    • even auto-generating summaries and deleting the transcripts themselves for a full end-to-end solution
    • drop me a comment here (or in Github) if you have any other ideas for improvements

    If you’re facing similar constraints – need transcription, can’t use cloud services, don’t control the meetings – this might help. Or inspire you to build your own version.

    Disclaimer: I’ve only “tested” this lightly today on a few meetings, so your mileage may vary, and do your own due diligence as always.

    Cheers,
    Dave

  • Interesting times

    Interesting times

    Well, we are certainly living in interesting times. The world seems to have gone to hell in a hand-basket over the last few weeks, doesn’t it? As the world is struggling to cope with finding a “new normal” amidst COVID-19 chaos, I thought I’d jot down a few rambling thoughts from my neck of the woods…

    Coping with the shift to working from home (remote working)

    We here at Timely are probably in a very fortunate position in that we already work remotely, and mostly from home at that. So when the various countries that we have staff in (NZ, AU, UK) went into lock-down, we didn’t really have to change too much about the way that we work.

    For a lot of other people, this whole “working from home” thing is new and scary. I get that. I remember moving from never really having worked from home almost 6 years ago, to working at Timely where we were 100% work from home. It took me months to get settled, and to find my “rhythm”. So the single biggest piece of advice I have here is not to sweat the small stuff, and don’t put too much pressure on yourself to be getting back up to 100% effectiveness too quickly.

    I won’t reiterate what a million other websites have already covered off (in terms of “top tips for working from home“, etc), but the few big ones for me are:

    Setting aside a dedicated work space

    Whether a spare room, a desk, a dining room table, whatever… make it your spot where you do work, and that’s all. Try not to use it for out of hours stuff like gaming, YouTube browsing, etc. Otherwise things tend to blur and you’ll either end up never “unplugging” from your work, or you’ll catch yourself watching 6 hours of cat videos on YouTube when you should be preparing those TPS reports.

    Change your expectations around communication

    When working in an office you get what I think of as real-time unconscious information all the time. Want to know if your colleague is available for a chat, just look up. Overhearing progress updates from a neighboring team’s morning stand-up. Seeing that a team-mate is busy working on that script you need for a deploy tomorrow.

    All of that goes out the window when working in isolation. You (and your team) need to get used to communicating more frequently and specifically. It won’t come naturally at first. You might feel like you’re “nagging” people, or interrupting them too often, but good communication will avoid frustrations down the line where you’re being held up because you’re not sure if someone else is working on what you need right away, or taking the dog for a walk instead.

    We use Slack a lot, but don’t rely on written comms too much (whether Slack, email, etc) – get used to jumping into video calls, even just for unscheduled 2-5 minutes chats throughout the day. Think in terms of where you were normally just lean over your desk to ask a coworker what they think about using this framework versus that one, or whether they’ve heard of the newest changes to the SaaS product you’re rolling out. Just spin up a call – it’s much quicker than typing out paragraphs in Slack, and gives you an excuse to chat to someone, even if just a little bit.

    Managing teams remotely

    Leading a team in this environment is definitely more challenging than doing so in a collocated work-space. A lot of what I said above about communication being critical is even more applicable when leading a remote team. Don’t discount the value of continuing things that your team might be used to doing in the office, such as morning stand-ups, maybe a social catch-up over lunch, or a beer at the end of the day, etc. All of these can be done remotely over a Hangout or Zoom meeting. It may be awkward at first. No, it will be awkward, but embrace the awkwardness and push through it.

    Over the past week or two my team have actually gone from just our normal 9am virtual stand-up each day, to having a second catch-up each afternoon as well. It’s not as structured as the morning stand-up, and is more just for chatting about how everyone’s days have gone, and usually devolves into poor attempts at joke telling, etc. All of this doesn’t have to take a huge amount of time. Some days when we’re “just not feeling it” they could be over in 5 minutes, and other days they’re 20 minutes of almost-crying-from-laughter – which is just what we all need a bit of at this time.

    As a wider company we’ve also become more active in our social channels in Slack; these are channels related to DIY, gardening, beer brewing, the gym, Lego, and a million other niche topics. Friday afternoons also tend to be more active, with various groups spinning up video calls where people can grab a beer/wine/water and have a bit of a virtual-social catch-up to close off the week. These have always been super-awkward when we’ve tried them in the past, but strangely now that they’re all we’ve got, they’re actually great fun! The trick is to limit them to small-ish groups (<12 people) and for someone to be ready with a few conversation starters if those awkward silences start creeping in. Give it a go and comment with your results, or any other tips you discover along the way. 🙂

    Remember that people are dealing with a lot right now, so on some days your team will be firing on all cylinders, and on other days people will be withdrawn, or overly sensitive, passive aggressive, etc. The best you can do is try to gauge this as quickly as possible and then tailor your interactions to suit.

    Dealing with isolation and social pressure

    All the above stuff talks about dealing with working from home, or managing a team during these trying times – but the most important piece of this puzzle is making sure you’re looking after yourself. This is something I definitely battle with.

    Before having to isolate I enjoyed going to the gym every day. My partner is an IFBB bikini competitor who was due to compete just a few weeks ago (before it was cancelled due to COVID related travel restrictions), and I’ve dabbled in powerlifting for years now. So naturally my social media is now filled with people going on about home workouts, staying motivated, videos of people doing all these awesome, clever workouts, or going for runs… you get the idea. At the same time I feel zero motivation to exercise, and am stuck inside eating too much food (and probably drinking more whisky than usual), which makes me feel even worse.

    Similarly there’s a lot of pressure at the moment to use this “free time” you now have to better yourself. Every other person seems to be earning their MBA or getting a bloody masters in something from some online university… and here I am still trying to finish that 6 hour Pluralsight course I’ve been busy with for the past year. Never mind the guitar I’m trying to learn that I haven’t picked up in 2 months…

    On top of this (as if it wasn’t enough) there’s stress and uncertainty about our jobs, the welfare of family and friends (I’ve got family in South Africa, which is definitely less equipped to deal with this than New Zealand is), making sure our kids are coping (and still learning while out of school), and a multitude of other things.

    So I think the answer is to just not give a f__k… for now. There are probably more eloquent ways of phrasing it – but basically I think that, just like it takes time to get used to working from home, it’s going to take time to adjust to everything else that’s changed in our daily routines. So I’m not going to worry about putting on a few extra kg’s, or my pile of unread books not shrinking as quickly as I’d like – in the same way that you shouldn’t worry about your work productivity taking a hit when you first start working from home.

    Each day I’ll try to stick to a routine as much as possible. When it works – great! But when it doesn’t, there’s always tomorrow to try again.

    Dave

  • Cloud Engineers – I want you!

    Cloud Engineers – I want you!

    I’ve been wanting to resurrect this blog for a while, so what better way to do so than advertising the fact that I need some help at work? 🙂

    Here at Timely I head up the Platform team. We’re currently a team of 5 (including myself) covering areas such as internal support, reporting, data and databases, devops and security, and performance and tooling.

    We’ve done a kick-ass job so far (if I do say so myself), but that backlog is starting to grow faster than we can knock out tasks. There are operational and DevOps related projects on the back-burner, and a bunch of security related enhancements we want to make too.

    So we’re going to plug a few gaps in the team by finding someone to look after our core infrastructure (based almost exclusively on Microsoft Azure). This person will also kick it up a few notches by finishing our IaC implementation (Infrastructure-as-Code, using Terraform) and automating away as much toil as possible. There’ll also be projects like improving our DR capabilities, taking our operational monitoring platforms to the next level, and a bunch more!

    So if this sounds like you, or someone you know an awesome Cloud Engineer, please get in touch! https://www.gettimely.com/careers/cloud-engineer/

  • PowerShell script to update Azure firewall rule

    PowerShell script to update Azure firewall rule

    *Updated 2017-06-28

    I’ve recently moved house, and as a result had to change my broadband plan from cable to ADSL (sad face). This also means I’ve gone from having a fixed IP address to a dynamically assigned one. Usually, this wouldn’t be a problem, except when it comes to connecting to the several Azure servers that I manage on a daily basis. Now I need to use the Azure Portal to manually change each server’s firewall settings at least once or twice a week. Painfull…

    So I quickly threw together this PS script to do the job for me and thought others out there might find it useful too.

    How’s it work?

    The script accepts an array of Azure SQL Server names, finds your external IP address using ipinfo.io, and then loops through the list of servers. You’ll need to provide a default rule name or modify the function call to pass it in (maybe include it in the array if it’s different for each server?).

    It then checks the current IP address of the specified rule and, if it’s different to your external IP address, updates the firewall rule for you. #Magic

    Import-Module SQLPS -DisableNameChecking 
    Import-Module Azure 
    
    # Run Get-AzurePublishSettingsFile first to download the publish settings file for your Azure subscription
    # Full instructions here: https://docs.microsoft.com/en-us/powershell/module/azure/get-azurepublishsettingsfile?view=azuresmps-4.0.0
    
    Import-AzurePublishSettingsFile "C:\My_oresome_path\Sweet-as-publish-settings-file.publishsettings" # &lt;-- put the path to your publish settings file here
    
    # Now just add your server names to this array... or get fancy and look them up somehow, 
    # whether from a simple text file or something more exotic.
    [array]$AzureServers = @('servername01','servername02','servername03','servername04'); 
    
    # Just a little function to get your current external/public IP address
    function Get-MyIpAddress
    {
        $ip = Invoke-RestMethod http://ipinfo.io/json | Select -exp ip
        return $ip;
    }
    
    # This function does the work of changing the whitelist if necessary
    function Update-MyAzureFirewallRule 
    {
        Param (
            [Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelinebyPropertyName=$True)]
            [string]$ServerName,
            [string]$RuleName = 'Put_Your_Rule_Name_Here',
            [string]$IpAddress
            )
    
        # Gets the current rule (so we can see what the IP address is currently set to)
        $CurrentRule = Get-AzureSqlDatabaseServerFirewallRule -RuleName $RuleName -ServerName $ServerName; 
        $CurrentIp = $CurrentRule.StartIpAddress
    
        # If your current IP doesn't match what's in the whitelist, then update it
        if ($CurrentIp -ne $IpAddress)
        {
            Write-Host "Setting firewall rule '$RuleName' on server '$ServerName' to IP address '$IpAddress' (was '$CurrentIp')..."
            Set-AzureSqlDatabaseServerFirewallRule -StartIPAddress $IpAddress -EndIPAddress $IpAddress -RuleName $RuleName -ServerName $ServerName;
        }
        
    }
    
    if ($IpAddress = Get-MyIpAddress)
    {
        Write-Host "Your IP address is $IpAddress"
    
        foreach ($s in $AzureServers) 
        {
            Update-MyAzureFirewallRule -ServerName $s -IpAddress $IpAddress;
        }
    }

    This post provided the inspiration, which I then tweaked it to suit my needs. Like I said; it’s quick-and-dirty, so use at your own risk. 😉  I’m no PowerShell guru either, so feel free to let me know if you improve on it.

    Cheers,
    Dave

  • SQL script of HTML status codes

    HTML Codes SQL Script

    Bit of a random post; I couldn’t find anything similar online, so created this script and thought I’d share it in case anyone randomly needs a lookup/dimension table of HTML status codes. All pulled directly from Wikipedia. 🙂