Compare commits

..
4 Commits
Author SHA1 Message Date
sirrowandClaude Opus 5 e2f8036682 count links as a fixed length against the tweet limit
Twitter rewrites every URL to a t.co address of a fixed 23 characters,
so counting the raw characters rejected messages Twitter would have
accepted: 100 characters of text plus a 50 character YouTube link
counted as 150 and was refused, where Twitter sees 123.

The punctuation trimmed off the end of a link keeps counting as ordinary
text. Only links written with a scheme are recognised; Twitter also
shortens bare hosts like example.com/foo, but detecting those without
mistaking Node.js for a link is not worth it here, and counting them in
full errs toward refusing rather than posting something too long.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 12:00:22 +09:00
sirrowandClaude Opus 5 fd14c07ae0 expand a preview card for YouTube links
A message containing a YouTube link now carries a preview card. Discord
and X build their own card from the URL in the text, so the work is on
the bluesky side: the new preview.go asks YouTube's oEmbed endpoint for
the title, channel and thumbnail and posts them as an
app.bsky.embed.external.

Links are found by pulling candidates out of the text and parsing them
with net/url rather than by matching a URL shaped regexp, so a host like
youtube.com.example.invalid is not mistaken for the real thing.

Bluesky allows one embed per post, so a message with both attachments
and a link keeps the attachments. A failed oEmbed lookup only costs the
card, not the post.

Downloading the thumbnail wants the attachment download path, so it is
split into fetch and downloadImage, and both now go through a client
with a timeout: the event loop is single threaded and a hung request
would stall every later message.

The Dockerfile built main.go by name, which stops working now that
package main spans two files; it builds the package instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 12:00:22 +09:00
sirrowandClaude Opus 5 1c2ab29404 mirror Discord replies as replies on X and bluesky
Replying to a message on the watched channel now posts the reply as a
reply to the posts that message produced, instead of as a standalone
post.

Discord tells us the parent through MessageReference, and the store maps
it back to the Ref each output returned; that Ref rides along on the new
Post.ReplyTo. X takes it as in_reply_to_tweet_id. Bluesky needs the
thread root as well as the parent, so a Ref now carries the root it was
posted under and a post that is not itself a reply acts as its own root.

Replying to something we never distributed, an error notice from the bot
for instance, still posts normally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 12:00:22 +09:00
sirrowandClaude Opus 5 067c9252c9 mirror Discord deletions to X and bluesky
Deleting a message on the watched channel now deletes the posts it
produced. Making that possible reshapes how a message reaches an output:

  - OutputInterface takes an output.Post and returns an output.Ref, the
    identifier the platform gave the new post (tweet ID for X, URI and
    CID for bluesky). It also gained Delete(Ref).
  - The discord package no longer forwards raw MessageCreate values but
    a discord.Event carrying the kind of change, so the MessageDelete
    handler can use the same channel. Deleted messages arrive without an
    author, so only the channel ID can be filtered on.
  - The new store package remembers Discord message ID -> per output Ref.
    Set POST_STORE to a path to keep that mapping across restarts;
    without it the mapping is memory only and a restart makes older
    messages undeletable. It holds the most recent 1000 messages.

main.go grows a distributor type to hold the outputs and the store
rather than threading them through the event loop.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-26 12:00:22 +09:00
14 changed files with 1069 additions and 80 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ WORKDIR tweetdistributor
RUN mkdir -p /build RUN mkdir -p /build
RUN go mod tidy RUN go mod tidy
RUN go build -a -tags "netgo" -tags timetzdata -installsuffix netgo -ldflags="-s -w -extldflags \"-static\"" -o=/build/tweetdistributor main.go RUN go build -a -tags "netgo" -tags timetzdata -installsuffix netgo -ldflags="-s -w -extldflags \"-static\"" -o=/build/tweetdistributor .
FROM alpine:3.22 FROM alpine:3.22
+51 -9
View File
@@ -6,34 +6,76 @@ import (
"github.com/bwmarrin/discordgo" "github.com/bwmarrin/discordgo"
) )
type discord struct { // EventKind tells apart the things that can happen to a watched message.
type EventKind int
const (
MessageCreated EventKind = iota
MessageDeleted
)
// Event is a change on the watched channel that the distributor has to mirror.
type Event struct {
Kind EventKind
MessageID string
Content string
Attachments []*discordgo.MessageAttachment
// ReplyToID is the ID of the message this one replies to, empty when the
// message is not a reply.
ReplyToID string
}
type Client struct {
Token string Token string
ChannelID string ChannelID string
dgsession *discordgo.Session dgsession *discordgo.Session
} }
func Discord(token string, channelId string) *discord { func Discord(token string, channelId string) *Client {
return &discord{ return &Client{
Token: token, Token: token,
ChannelID: channelId, ChannelID: channelId,
} }
} }
func (d *discord) BeginRead(tweetchannel chan<- *discordgo.MessageCreate) { func (d *Client) BeginRead(eventchannel chan<- Event) {
d.read(tweetchannel) d.read(eventchannel)
} }
func (d *discord) read(tweetchannel chan<- *discordgo.MessageCreate) { func (d *Client) read(eventchannel chan<- Event) {
dgsession, err := discordgo.New("Bot " + d.Token) dgsession, err := discordgo.New("Bot " + d.Token)
if err != nil { if err != nil {
return return
} }
dgsession.AddHandler(func(s *discordgo.Session, m *discordgo.MessageCreate) { dgsession.AddHandler(func(s *discordgo.Session, m *discordgo.MessageCreate) {
if m.ChannelID == d.ChannelID && m.Author.ID != s.State.User.ID { if m.ChannelID != d.ChannelID || m.Author.ID == s.State.User.ID {
tweetchannel <- m return
}
event := Event{
Kind: MessageCreated,
MessageID: m.ID,
Content: m.Content,
Attachments: m.Attachments,
}
if m.MessageReference != nil {
event.ReplyToID = m.MessageReference.MessageID
}
eventchannel <- event
})
// Deleted messages arrive without an author, so the channel is the only
// thing to filter on; messages we never distributed are dropped later.
dgsession.AddHandler(func(s *discordgo.Session, m *discordgo.MessageDelete) {
if m.ChannelID != d.ChannelID {
return
}
eventchannel <- Event{
Kind: MessageDeleted,
MessageID: m.ID,
} }
}) })
d.dgsession = dgsession d.dgsession = dgsession
errOpen := dgsession.Open() errOpen := dgsession.Open()
@@ -45,6 +87,6 @@ func Read() string {
return "discord" return "discord"
} }
func (d *discord) Write(str string) { func (d *Client) Write(str string) {
d.dgsession.ChannelMessageSend(d.ChannelID, str) d.dgsession.ChannelMessageSend(d.ChannelID, str)
} }
+228
View File
@@ -0,0 +1,228 @@
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/bluesky-social/indigo v0.0.0-20250313000755-d9a74f690c90 h1:lSo7V6ut0FrLmFyv3TfvXrqwcPNDXRC1hNvS04pLgfA=
github.com/bluesky-social/indigo v0.0.0-20250313000755-d9a74f690c90/go.mod h1:NVBwZvbBSa93kfyweAmKwOLYawdVHdwZ9s+GZtBBVLA=
github.com/bwmarrin/discordgo v0.28.1 h1:gXsuo2GBO7NbR6uqmrrBDplPUx2T3nzu775q/Rd1aG4=
github.com/bwmarrin/discordgo v0.28.1/go.mod h1:NJZpH+1AfhIcyQsPeuBKsUtYrRnjkyu0kIVMCHkZtRY=
github.com/carlmjohnson/versioninfo v0.22.5 h1:O00sjOLUAFxYQjlN/bzYTuZiS0y6fWDQjMRvwtKgwwc=
github.com/carlmjohnson/versioninfo v0.22.5/go.mod h1:QT9mph3wcVfISUKd0i9sZfVrPviHuSF+cUtLjm2WSf8=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.1 h1:pKouT5E8xu9zeFC39JXRDukb6JFQPXM5p5I91188VAQ=
github.com/go-logr/logr v1.4.1/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-hclog v0.9.2 h1:CG6TE5H9/JXsFWJCfoIVpKFIkFe6ysEuHirp4DxCsHI=
github.com/hashicorp/go-hclog v0.9.2/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ=
github.com/hashicorp/go-retryablehttp v0.7.5 h1:bJj+Pj19UZMIweq/iie+1u5YCdGrnxCT9yvm0e+Nd5M=
github.com/hashicorp/go-retryablehttp v0.7.5/go.mod h1:Jy/gPYAdjqffZ/yFGCFV2doI5wjtH1ewM9u8iYVjtX8=
github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=
github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
github.com/ipfs/go-block-format v0.2.0 h1:ZqrkxBA2ICbDRbK8KJs/u0O3dlp6gmAuuXUJNiW1Ycs=
github.com/ipfs/go-block-format v0.2.0/go.mod h1:+jpL11nFx5A/SPpsoBn6Bzkra/zaArfSmsknbPMYgzM=
github.com/ipfs/go-cid v0.4.1 h1:A/T3qGvxi4kpKWWcPC/PgbvDA2bjVLO7n4UeVwnbs/s=
github.com/ipfs/go-cid v0.4.1/go.mod h1:uQHwDeX4c6CtyrFwdqyhpNcxVewur1M7l7fNU7LKwZk=
github.com/ipfs/go-datastore v0.6.0 h1:JKyz+Gvz1QEZw0LsX1IBn+JFCJQH4SJVFtM4uWU0Myk=
github.com/ipfs/go-datastore v0.6.0/go.mod h1:rt5M3nNbSO/8q1t4LNkLyUwRs8HupMeN/8O4Vn9YAT8=
github.com/ipfs/go-detect-race v0.0.1 h1:qX/xay2W3E4Q1U7d9lNs1sU9nvguX0a7319XbyQ6cOk=
github.com/ipfs/go-detect-race v0.0.1/go.mod h1:8BNT7shDZPo99Q74BpGMK+4D8Mn4j46UU0LZ723meps=
github.com/ipfs/go-ipfs-blockstore v1.3.1 h1:cEI9ci7V0sRNivqaOr0elDsamxXFxJMMMy7PTTDQNsQ=
github.com/ipfs/go-ipfs-blockstore v1.3.1/go.mod h1:KgtZyc9fq+P2xJUiCAzbRdhhqJHvsw8u2Dlqy2MyRTE=
github.com/ipfs/go-ipfs-ds-help v1.1.1 h1:B5UJOH52IbcfS56+Ul+sv8jnIV10lbjLF5eOO0C66Nw=
github.com/ipfs/go-ipfs-ds-help v1.1.1/go.mod h1:75vrVCkSdSFidJscs8n4W+77AtTpCIAdDGAwjitJMIo=
github.com/ipfs/go-ipfs-util v0.0.3 h1:2RFdGez6bu2ZlZdI+rWfIdbQb1KudQp3VGwPtdNCmE0=
github.com/ipfs/go-ipfs-util v0.0.3/go.mod h1:LHzG1a0Ig4G+iZ26UUOMjHd+lfM84LZCrn17xAKWBvs=
github.com/ipfs/go-ipld-cbor v0.1.0 h1:dx0nS0kILVivGhfWuB6dUpMa/LAwElHPw1yOGYopoYs=
github.com/ipfs/go-ipld-cbor v0.1.0/go.mod h1:U2aYlmVrJr2wsUBU67K4KgepApSZddGRDWBYR0H4sCk=
github.com/ipfs/go-ipld-format v0.6.0 h1:VEJlA2kQ3LqFSIm5Vu6eIlSxD/Ze90xtc4Meten1F5U=
github.com/ipfs/go-ipld-format v0.6.0/go.mod h1:g4QVMTn3marU3qXchwjpKPKgJv+zF+OlaKMyhJ4LHPg=
github.com/ipfs/go-log v1.0.5 h1:2dOuUCB1Z7uoczMWgAyDck5JLb72zHzrMnGnCNNbvY8=
github.com/ipfs/go-log v1.0.5/go.mod h1:j0b8ZoR+7+R99LD9jZ6+AJsrzkPbSXbZfGakb5JPtIo=
github.com/ipfs/go-log/v2 v2.1.3/go.mod h1:/8d0SH3Su5Ooc31QlL1WysJhvyOTDCjcCZ9Axpmri6g=
github.com/ipfs/go-log/v2 v2.5.1 h1:1XdUzF7048prq4aBjDQQ4SL5RxftpRGdXhNRwKSAlcY=
github.com/ipfs/go-log/v2 v2.5.1/go.mod h1:prSpmC1Gpllc9UYWxDiZDreBYw7zp4Iqp1kOLU9U5UI=
github.com/ipfs/go-metrics-interface v0.0.1 h1:j+cpbjYvu4R8zbleSs36gvB7jR+wsL2fGD6n0jO4kdg=
github.com/ipfs/go-metrics-interface v0.0.1/go.mod h1:6s6euYU4zowdslK0GKHmqaIZ3j/b/tL7HTWtJ4VPgWY=
github.com/jbenet/go-cienv v0.1.0/go.mod h1:TqNnHUmJgXau0nCzC7kXWeotg3J9W34CUv5Djy1+FlA=
github.com/jbenet/goprocess v0.1.4 h1:DRGOFReOMqqDNXwW70QkacFW0YN9QnwLV0Vqk+3oU0o=
github.com/jbenet/goprocess v0.1.4/go.mod h1:5yspPrukOVuOLORacaBi858NqyClJPQxYZlqdZVfqY4=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/michimani/gotwi v0.18.2 h1:2eU7Lunc5eMyOM/XITz5ahsDNhUY9D363QaR3DddjfQ=
github.com/michimani/gotwi v0.18.2/go.mod h1:yz1cyV/30Uy/KGQyN8BVfXFPt/63Imzonykny8/SMi0=
github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=
github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/multiformats/go-base32 v0.1.0 h1:pVx9xoSPqEIQG8o+UbAe7DNi51oej1NtK+aGkbLYxPE=
github.com/multiformats/go-base32 v0.1.0/go.mod h1:Kj3tFY6zNr+ABYMqeUNeGvkIC/UYgtWibDcT0rExnbI=
github.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0=
github.com/multiformats/go-base36 v0.2.0/go.mod h1:qvnKE++v+2MWCfePClUEjE78Z7P2a1UV0xHgWc0hkp4=
github.com/multiformats/go-multibase v0.2.0 h1:isdYCVLvksgWlMW9OZRYJEa9pZETFivncJHmHnnd87g=
github.com/multiformats/go-multibase v0.2.0/go.mod h1:bFBZX4lKCA/2lyOFSAoKH5SS6oPyjtnzK/XTFDPkNuk=
github.com/multiformats/go-multihash v0.2.3 h1:7Lyc8XfX/IY2jWb/gI7JP+o7JEq9hOa7BFvVU9RSh+U=
github.com/multiformats/go-multihash v0.2.3/go.mod h1:dXgKXCXjBzdscBLk9JkjINiEsCKRVch90MdaGiKsvSM=
github.com/multiformats/go-varint v0.0.7 h1:sWSGR+f/eu5ABZA2ZpYKBILXTTs9JWpdEM/nEGOHFS8=
github.com/multiformats/go-varint v0.0.7/go.mod h1:r8PUYw/fD/SjBCiKOoDlGF6QawOELpZAu9eioSos/OU=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/polydawn/refmt v0.89.1-0.20221221234430-40501e09de1f h1:VXTQfuJj9vKR4TCkEuWIckKvdHFeJH/huIFJ9/cXOB0=
github.com/polydawn/refmt v0.89.1-0.20221221234430-40501e09de1f/go.mod h1:/zvteZs/GwLtCgZ4BL6CBsk9IKIlexP43ObX9AxTqTw=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/smartystreets/assertions v1.2.0 h1:42S6lae5dvLc7BrLu/0ugRtcFVjoJNMC/N3yZFZkDFs=
github.com/smartystreets/assertions v1.2.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo=
github.com/smartystreets/goconvey v1.7.2 h1:9RBaZCeXEQ3UselpuwUQHltGVXvdwm6cv1hgR6gDIPg=
github.com/smartystreets/goconvey v1.7.2/go.mod h1:Vw0tHAZW6lzCRk3xgdin6fKYcG+G3Pg9vgXWeJpQFMM=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/urfave/cli v1.22.10/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0 h1:GDDkbFiaK8jsSDJfjId/PEGEShv6ugrt4kYsC5UIDaQ=
github.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0/go.mod h1:x6AKhvSSexNrVSrViXSHUEbICjmGXhtgABaHIySUSGw=
github.com/whyrusleeping/cbor-gen v0.2.1-0.20241030202151-b7a6831be65e h1:28X54ciEwwUxyHn9yrZfl5ojgF4CBNLWX7LR0rvBkf4=
github.com/whyrusleeping/cbor-gen v0.2.1-0.20241030202151-b7a6831be65e/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 h1:aFJWCqJMNjENlcleuuOkGAPH82y0yULBScfXcIEdS24=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1/go.mod h1:sEGXWArGqc3tVa+ekntsN65DmVbVeW+7lTKTjZF3/Fo=
go.opentelemetry.io/otel v1.21.0 h1:hzLeKBZEL7Okw2mGzZ0cc4k/A7Fta0uoPgaJCr8fsFc=
go.opentelemetry.io/otel v1.21.0/go.mod h1:QZzNPQPm1zLX4gZK4cMi+71eaorMSGT3A4znnUvNNEo=
go.opentelemetry.io/otel/metric v1.21.0 h1:tlYWfeo+Bocx5kLEloTjbcDwBuELRrIFxwdQ36PlJu4=
go.opentelemetry.io/otel/metric v1.21.0/go.mod h1:o1p3CA8nNHW8j5yuQLdc1eeqEaPfzug24uvsyIEJRWM=
go.opentelemetry.io/otel/trace v1.21.0 h1:WD9i5gzvoUPuXIXH24ZNBudiarZDKuekPqi/E8fpfLc=
go.opentelemetry.io/otel/trace v1.21.0/go.mod h1:LGbsEB0f9LGjN+OZaQQ26sohbOmiMR+BaslueVtS/qQ=
go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.1.11-0.20210813005559-691160354723/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
go.uber.org/goleak v1.2.0/go.mod h1:XJYK+MuIchqpmGmUSAzotztawfKvYLUIgg7guXrwVUo=
go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU=
go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA=
go.uber.org/zap v1.16.0/go.mod h1:MA8QOfq0BHJwdXa996Y4dYkAqRKB8/1K1QMMZVaNZjQ=
go.uber.org/zap v1.19.1/go.mod h1:j3DNczoxDZroyBnOT1L/Q79cfUMGZxlv/9dzN7SM1rI=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/image v0.43.0 h1:FLxcP4ec2350nTfOC8ysKtqYSIFbk/QGjw1ZHNP4tsY=
golang.org/x/image v0.43.0/go.mod h1:rrpelvGFt+kLPAjPM4HeWPgrl0FtafueU//e5N0qk/Q=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 h1:+cNy6SZtPcJQH3LJVLOSmiC7MMxXNOb3PU/VUEz+EhU=
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
lukechampine.com/blake3 v1.2.1 h1:YuqqRuaqsGV71BV/nm9xlI0MKUv4QC54jQnBChWbGnI=
lukechampine.com/blake3 v1.2.1/go.mod h1:0OFRp7fBtAylGVCO40o87sbupkyIGgbpv1+M1k1LM6k=
+173 -47
View File
@@ -11,9 +11,12 @@ import (
"net/http" "net/http"
"os" "os"
"path" "path"
"regexp"
"strings" "strings"
"time"
"tweetdistributor/discord" "tweetdistributor/discord"
"tweetdistributor/output" "tweetdistributor/output"
"tweetdistributor/store"
"unicode/utf8" "unicode/utf8"
"github.com/bwmarrin/discordgo" "github.com/bwmarrin/discordgo"
@@ -27,6 +30,33 @@ const maxImagesPerPost = 4
// Bluesky rejects blobs over 2,000,000 bytes; Twitter allows up to 5MB. // Bluesky rejects blobs over 2,000,000 bytes; Twitter allows up to 5MB.
const maxImageBytes = 2_000_000 const maxImageBytes = 2_000_000
// urlLength is what a link costs against maxTweetLength: Twitter rewrites
// every URL to a t.co address of this fixed length, however long the original
// was, so counting the raw characters would reject messages it would accept.
const urlLength = 23
// urlPattern picks candidate links out of message text; each one is parsed
// properly before it is judged to be a YouTube link.
var urlPattern = regexp.MustCompile(`https?://[^\s<>"']+`)
// trimURL drops the punctuation that ends the sentence rather than the URL.
func trimURL(match string) string {
return strings.TrimRight(match, ".,!?、。)]}>")
}
// tweetLength counts a message the way Twitter does, charging every link a
// fixed length instead of its actual one.
func tweetLength(content string) int {
length := utf8.RuneCountInString(content)
for _, match := range urlPattern.FindAllString(content, -1) {
// The trimmed punctuation is still ordinary text and keeps counting.
length += urlLength - utf8.RuneCountInString(trimURL(match))
}
return length
}
var httpClient = &http.Client{Timeout: 30 * time.Second}
// shrinkImage re-encodes (and if necessary downscales) an image until it // shrinkImage re-encodes (and if necessary downscales) an image until it
// fits within maxImageBytes. Images already small enough pass through // fits within maxImageBytes. Images already small enough pass through
// untouched. // untouched.
@@ -68,29 +98,44 @@ func shrinkImage(img output.Image) (output.Image, error) {
return output.Image{}, fmt.Errorf("%s could not be shrunk below %d bytes", img.Filename, maxImageBytes) return output.Image{}, fmt.Errorf("%s could not be shrunk below %d bytes", img.Filename, maxImageBytes)
} }
// fetch GETs url and returns its body.
func fetch(url string) ([]byte, error) {
resp, err := httpClient.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
data, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("status %s", resp.Status)
}
return data, nil
}
// downloadImage fetches an image and shrinks it to a postable size.
func downloadImage(url, filename, contentType string) (output.Image, error) {
data, err := fetch(url)
if err != nil {
return output.Image{}, fmt.Errorf("downloading %s: %w", filename, err)
}
return shrinkImage(output.Image{
Data: data,
ContentType: contentType,
Filename: filename,
})
}
func downloadImages(attachments []*discordgo.MessageAttachment) ([]output.Image, error) { func downloadImages(attachments []*discordgo.MessageAttachment) ([]output.Image, error) {
var images []output.Image var images []output.Image
for _, attachment := range attachments { for _, attachment := range attachments {
if !strings.HasPrefix(attachment.ContentType, "image/") { if !strings.HasPrefix(attachment.ContentType, "image/") {
continue continue
} }
resp, err := http.Get(attachment.URL) img, err := downloadImage(attachment.URL, attachment.Filename, attachment.ContentType)
if err != nil {
return nil, fmt.Errorf("downloading %s: %w", attachment.Filename, err)
}
data, err := io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("downloading %s: %w", attachment.Filename, err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("downloading %s: status %s", attachment.Filename, resp.Status)
}
img, err := shrinkImage(output.Image{
Data: data,
ContentType: attachment.ContentType,
Filename: attachment.Filename,
})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -99,11 +144,113 @@ func downloadImages(attachments []*discordgo.MessageAttachment) ([]output.Image,
return images, nil return images, nil
} }
// distributor mirrors what happens on the Discord channel to every output.
type distributor struct {
d *discord.Client
outputs []output.OutputInterface
store *store.Store
}
// reportf logs an error and echoes it back into the Discord channel.
func (dist *distributor) reportf(format string, args ...any) {
errstr := fmt.Sprintf(format, args...)
fmt.Fprintln(os.Stderr, errstr)
dist.d.Write(errstr)
}
// created posts a new Discord message to every output, as a reply when the
// Discord message itself was a reply to something we already distributed.
func (dist *distributor) created(event discord.Event) {
if length := tweetLength(event.Content); length > maxTweetLength {
dist.reportf("Error: message is %d characters counting each link as %d, exceeding the %d character limit; not posted", length, urlLength, maxTweetLength)
return
}
images, err := downloadImages(event.Attachments)
if err != nil {
dist.reportf("Error: %s; not posted", err)
return
}
if len(images) > maxImagesPerPost {
dist.reportf("Error: %d images attached, exceeding the limit of %d; not posted", len(images), maxImagesPerPost)
return
}
var preview *output.Preview
if videoURL := findYouTubeURL(event.Content); videoURL != "" {
preview, err = youtubePreview(videoURL)
if err != nil {
// The post is still worth making without its card.
fmt.Fprintln(os.Stderr, err)
}
}
// A reply to a message we never distributed becomes a top level post.
var parents store.Refs
if event.ReplyToID != "" {
parents, _ = dist.store.Get(event.ReplyToID)
}
refs := store.Refs{}
for _, out := range dist.outputs {
post := output.Post{
Text: event.Content,
Images: images,
Preview: preview,
}
if parent, ok := parents[out.GetName()]; ok && !parent.IsZero() {
post.ReplyTo = &parent
}
ref, err := out.Write(post)
if err != nil {
dist.reportf("%s Error: %s", out.GetName(), err)
continue
}
refs[out.GetName()] = ref
}
if len(refs) == 0 {
return
}
if err := dist.store.Put(event.MessageID, refs); err != nil {
dist.reportf("Error: could not remember the posts for this message: %s", err)
}
}
// deleted removes the posts that a now deleted Discord message produced.
func (dist *distributor) deleted(event discord.Event) {
refs, ok := dist.store.Get(event.MessageID)
if !ok {
return
}
for _, out := range dist.outputs {
ref, ok := refs[out.GetName()]
if !ok {
continue
}
if err := out.Delete(ref); err != nil {
dist.reportf("%s Error: could not delete the post: %s", out.GetName(), err)
}
}
if err := dist.store.Delete(event.MessageID); err != nil {
dist.reportf("Error: could not forget the posts for this message: %s", err)
}
}
func main() { func main() {
posts, err := store.New(os.Getenv("POST_STORE"))
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
d := discord.Discord(os.Getenv("DISCORD_TOKEN"), os.Getenv("DISCORD_CHANNEL")) d := discord.Discord(os.Getenv("DISCORD_TOKEN"), os.Getenv("DISCORD_CHANNEL"))
tweetchannel := make(chan *discordgo.MessageCreate, 1) eventchannel := make(chan discord.Event, 1)
d.BeginRead(tweetchannel) d.BeginRead(eventchannel)
d.Write("Tweetdistributor Started") d.Write("Tweetdistributor Started")
@@ -112,35 +259,14 @@ func main() {
outputs = append(outputs, output.TwitterOutput(os.Getenv("TW_ACCESS_TOKEN"), os.Getenv("TW_ACCESS_SECRET"))) outputs = append(outputs, output.TwitterOutput(os.Getenv("TW_ACCESS_TOKEN"), os.Getenv("TW_ACCESS_SECRET")))
outputs = append(outputs, output.BlueskyOutput(os.Getenv("BSKY_IDENTIFIER"), os.Getenv("BSKY_PASSWORD"))) outputs = append(outputs, output.BlueskyOutput(os.Getenv("BSKY_IDENTIFIER"), os.Getenv("BSKY_PASSWORD")))
for tweet := range tweetchannel { dist := &distributor{d: d, outputs: outputs, store: posts}
if length := utf8.RuneCountInString(tweet.Content); length > maxTweetLength {
errstr := fmt.Sprintf("Error: message is %d characters, exceeding the %d character limit; not posted", length, maxTweetLength)
fmt.Fprintln(os.Stderr, errstr)
d.Write(errstr)
continue
}
images, err := downloadImages(tweet.Attachments) for event := range eventchannel {
if err != nil { switch event.Kind {
errstr := fmt.Sprintf("Error: %s; not posted", err) case discord.MessageCreated:
fmt.Fprintln(os.Stderr, errstr) dist.created(event)
d.Write(errstr) case discord.MessageDeleted:
continue dist.deleted(event)
}
if len(images) > maxImagesPerPost {
errstr := fmt.Sprintf("Error: %d images attached, exceeding the limit of %d; not posted", len(images), maxImagesPerPost)
fmt.Fprintln(os.Stderr, errstr)
d.Write(errstr)
continue
}
for _, output := range outputs {
err := output.Write(tweet.Content, images)
if err != nil {
errstr := fmt.Sprintf("%s Error: %s", output.GetName(), err)
fmt.Fprintln(os.Stderr, errstr)
d.Write(errstr)
}
} }
} }
} }
+103
View File
@@ -0,0 +1,103 @@
package main
import (
"bytes"
"image"
"image/color"
"image/jpeg"
"math/rand"
"strings"
"testing"
"tweetdistributor/output"
"unicode/utf8"
)
// noiseJPEG returns a JPEG image of random noise, which compresses poorly
// and reliably exceeds maxImageBytes at large dimensions.
func noiseJPEG(t *testing.T, width, height int) []byte {
t.Helper()
rng := rand.New(rand.NewSource(1))
img := image.NewRGBA(image.Rect(0, 0, width, height))
for y := 0; y < height; y++ {
for x := 0; x < width; x++ {
img.Set(x, y, color.RGBA{uint8(rng.Intn(256)), uint8(rng.Intn(256)), uint8(rng.Intn(256)), 255})
}
}
var buf bytes.Buffer
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: 95}); err != nil {
t.Fatal(err)
}
return buf.Bytes()
}
func TestTweetLength(t *testing.T) {
tests := []struct {
name string
content string
want int
}{
{"no link", "こんにちは", 5},
{"long link", "https://www.youtube.com/watch?v=dQw4w9WgXcQ&list=PLabcdefghijklmnop", urlLength},
{"short link", "https://a.jp", urlLength},
{"link with text", "みてね " + strings.Repeat("x", 10) + " https://youtu.be/dQw4w9WgXcQ", 4 + 10 + 1 + urlLength},
{"two links", "https://a.jp https://b.jp", urlLength*2 + 1},
{"trailing punctuation still counts", "https://a.jp。", urlLength + 1},
{"multibyte path", "https://ja.wikipedia.org/wiki/日本語", urlLength},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := tweetLength(tt.content); got != tt.want {
t.Errorf("tweetLength(%q) = %d, want %d", tt.content, got, tt.want)
}
})
}
}
func TestTweetLengthAcceptsMessageOnlyLinksMakeTooLong(t *testing.T) {
// A message that the raw rune count would reject but Twitter accepts.
content := strings.Repeat("あ", 100) + " https://www.youtube.com/watch?v=dQw4w9WgXcQ&t=120s"
if raw := utf8.RuneCountInString(content); raw <= maxTweetLength {
t.Fatalf("test message should be %d raw characters, got %d", maxTweetLength+1, raw)
}
if got := tweetLength(content); got > maxTweetLength {
t.Errorf("tweetLength = %d, want it to fit within %d", got, maxTweetLength)
}
}
func TestShrinkImageLarge(t *testing.T) {
data := noiseJPEG(t, 3000, 3000)
if len(data) <= maxImageBytes {
t.Fatalf("test image should exceed maxImageBytes, got %d", len(data))
}
shrunk, err := shrinkImage(output.Image{Data: data, ContentType: "image/jpeg", Filename: "DSC_7277.jpg"})
if err != nil {
t.Fatal(err)
}
if len(shrunk.Data) > maxImageBytes {
t.Errorf("shrunk image is %d bytes, want <= %d", len(shrunk.Data), maxImageBytes)
}
if shrunk.ContentType != "image/jpeg" {
t.Errorf("ContentType = %q, want image/jpeg", shrunk.ContentType)
}
if shrunk.Filename != "DSC_7277.jpg" {
t.Errorf("Filename = %q, want DSC_7277.jpg", shrunk.Filename)
}
if _, _, err := image.Decode(bytes.NewReader(shrunk.Data)); err != nil {
t.Errorf("shrunk image is not decodable: %s", err)
}
}
func TestShrinkImageSmallPassthrough(t *testing.T) {
data := noiseJPEG(t, 100, 100)
img := output.Image{Data: data, ContentType: "image/png", Filename: "small.png"}
shrunk, err := shrinkImage(img)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(shrunk.Data, img.Data) || shrunk.ContentType != img.ContentType || shrunk.Filename != img.Filename {
t.Error("small image should pass through unchanged")
}
}
+107 -13
View File
@@ -3,7 +3,9 @@ package output
import ( import (
"bytes" "bytes"
"context" "context"
"errors"
"fmt" "fmt"
"strings"
"time" "time"
"github.com/bluesky-social/indigo/api/atproto" "github.com/bluesky-social/indigo/api/atproto"
@@ -13,6 +15,8 @@ import (
"github.com/bluesky-social/indigo/xrpc" "github.com/bluesky-social/indigo/xrpc"
) )
const blueskyCollection = "app.bsky.feed.post"
type blueskyoutput struct { type blueskyoutput struct {
identifier string identifier string
password string password string
@@ -27,7 +31,8 @@ func BlueskyOutput(identifier string, password string) *blueskyoutput {
return blueskyoutput return blueskyoutput
} }
func (bo *blueskyoutput) Write(str string, images []Image) error { // session logs in and returns a client authenticated as the configured account.
func (bo *blueskyoutput) session() (*xrpc.Client, error) {
cli := &xrpc.Client{ cli := &xrpc.Client{
Host: "https://bsky.social", Host: "https://bsky.social",
} }
@@ -39,7 +44,7 @@ func (bo *blueskyoutput) Write(str string, images []Image) error {
output, err := atproto.ServerCreateSession(context.TODO(), cli, input) output, err := atproto.ServerCreateSession(context.TODO(), cli, input)
if err != nil { if err != nil {
return err return nil, err
} }
cli.Auth = &xrpc.AuthInfo{ cli.Auth = &xrpc.AuthInfo{
AccessJwt: output.AccessJwt, AccessJwt: output.AccessJwt,
@@ -47,40 +52,129 @@ func (bo *blueskyoutput) Write(str string, images []Image) error {
Handle: output.Handle, Handle: output.Handle,
Did: output.Did, Did: output.Did,
} }
return cli, nil
}
post := &bsky.FeedPost{ func (bo *blueskyoutput) Write(post Post) (Ref, error) {
Text: str, cli, err := bo.session()
if err != nil {
return Ref{}, err
}
feedpost := &bsky.FeedPost{
Text: post.Text,
CreatedAt: time.Now().Format(util.ISO8601), CreatedAt: time.Now().Format(util.ISO8601),
Langs: []string{"ja"}, Langs: []string{"ja"},
} }
if len(images) > 0 { root := Ref{}
embedimages := make([]*bsky.EmbedImages_Image, 0, len(images)) if post.ReplyTo != nil && post.ReplyTo.URI != "" {
for _, img := range images { parent := &atproto.RepoStrongRef{
Uri: post.ReplyTo.URI,
Cid: post.ReplyTo.CID,
}
// A post that is itself a reply carries the thread root; one that is
// not is the root of its own thread.
rootref := &atproto.RepoStrongRef{
Uri: post.ReplyTo.RootURI,
Cid: post.ReplyTo.RootCID,
}
if rootref.Uri == "" {
rootref = parent
}
feedpost.Reply = &bsky.FeedPost_ReplyRef{
Parent: parent,
Root: rootref,
}
root = Ref{RootURI: rootref.Uri, RootCID: rootref.Cid}
}
// Bluesky allows a single embed, so attached images win over a link card.
switch {
case len(post.Images) > 0:
embedimages := make([]*bsky.EmbedImages_Image, 0, len(post.Images))
for _, img := range post.Images {
blob, err := atproto.RepoUploadBlob(context.TODO(), cli, bytes.NewReader(img.Data)) blob, err := atproto.RepoUploadBlob(context.TODO(), cli, bytes.NewReader(img.Data))
if err != nil { if err != nil {
return fmt.Errorf("uploading %s: %w", img.Filename, err) return Ref{}, fmt.Errorf("uploading %s: %w", img.Filename, err)
} }
embedimages = append(embedimages, &bsky.EmbedImages_Image{ embedimages = append(embedimages, &bsky.EmbedImages_Image{
Alt: img.Filename, Alt: img.Filename,
Image: blob.Blob, Image: blob.Blob,
}) })
} }
post.Embed = &bsky.FeedPost_Embed{ feedpost.Embed = &bsky.FeedPost_Embed{
EmbedImages: &bsky.EmbedImages{ EmbedImages: &bsky.EmbedImages{
Images: embedimages, Images: embedimages,
}, },
} }
case post.Preview != nil:
external := &bsky.EmbedExternal_External{
Uri: post.Preview.URL,
Title: post.Preview.Title,
Description: post.Preview.Description,
}
if thumb := post.Preview.Thumb; thumb != nil {
blob, err := atproto.RepoUploadBlob(context.TODO(), cli, bytes.NewReader(thumb.Data))
if err != nil {
return Ref{}, fmt.Errorf("uploading %s: %w", thumb.Filename, err)
}
external.Thumb = blob.Blob
}
feedpost.Embed = &bsky.FeedPost_Embed{
EmbedExternal: &bsky.EmbedExternal{
External: external,
},
}
} }
Recordinput := &atproto.RepoCreateRecord_Input{ Recordinput := &atproto.RepoCreateRecord_Input{
Collection: "app.bsky.feed.post", Collection: blueskyCollection,
Repo: cli.Auth.Did, // "matope.bsky.social" のDID Repo: cli.Auth.Did, // "matope.bsky.social" のDID
Record: &lexutil.LexiconTypeDecoder{Val: post}, Record: &lexutil.LexiconTypeDecoder{Val: feedpost},
}
record, recerr := atproto.RepoCreateRecord(context.TODO(), cli, Recordinput)
if recerr != nil {
return Ref{}, recerr
} }
_, recerr := atproto.RepoCreateRecord(context.TODO(), cli, Recordinput)
return recerr return Ref{
URI: record.Uri,
CID: record.Cid,
RootURI: root.RootURI,
RootCID: root.RootCID,
}, nil
}
func (bo *blueskyoutput) Delete(ref Ref) error {
rkey, err := recordKey(ref.URI)
if err != nil {
return err
}
cli, err := bo.session()
if err != nil {
return err
}
_, err = atproto.RepoDeleteRecord(context.TODO(), cli, &atproto.RepoDeleteRecord_Input{
Collection: blueskyCollection,
Repo: cli.Auth.Did,
Rkey: rkey,
})
return err
}
// recordKey pulls the rkey out of an at://did/collection/rkey URI.
func recordKey(uri string) (string, error) {
if uri == "" {
return "", errors.New("no record URI to delete")
}
idx := strings.LastIndex(uri, "/")
if idx < 0 || idx == len(uri)-1 {
return "", fmt.Errorf("malformed record URI %q", uri)
}
return uri[idx+1:], nil
} }
func (bo *blueskyoutput) GetName() string { func (bo *blueskyoutput) GetName() string {
+19
View File
@@ -0,0 +1,19 @@
package output
import "testing"
func TestRecordKey(t *testing.T) {
rkey, err := recordKey("at://did:plc:abc123/app.bsky.feed.post/3kqz7xyz")
if err != nil {
t.Fatal(err)
}
if rkey != "3kqz7xyz" {
t.Errorf("recordKey = %q, want 3kqz7xyz", rkey)
}
for _, uri := range []string{"", "at://did:plc:abc123/app.bsky.feed.post/", "nonsense"} {
if _, err := recordKey(uri); err == nil {
t.Errorf("recordKey(%q) returned no error", uri)
}
}
}
+36 -1
View File
@@ -6,7 +6,42 @@ type Image struct {
Filename string Filename string
} }
// Preview is a link preview card ("プレビュー窓") to attach to a post.
type Preview struct {
URL string
Title string
Description string
Thumb *Image
}
// Ref identifies a post an output has already published so it can later be
// replied to or deleted. The fields are output specific; only the ones the
// publishing output filled in are meaningful to it.
type Ref struct {
ID string `json:"id,omitempty"` // twitter: tweet ID
URI string `json:"uri,omitempty"` // bluesky: at:// URI of the record
CID string `json:"cid,omitempty"` // bluesky: record CID
RootURI string `json:"rooturi,omitempty"`
RootCID string `json:"rootcid,omitempty"`
}
// IsZero reports whether the ref points at nothing.
func (r Ref) IsZero() bool {
return r == Ref{}
}
// Post is a single message to distribute.
type Post struct {
Text string
Images []Image
Preview *Preview
// ReplyTo is the Ref this same output returned for the post being
// replied to, or nil for a top level post.
ReplyTo *Ref
}
type OutputInterface interface { type OutputInterface interface {
Write(string, []Image) error Write(Post) (Ref, error)
Delete(Ref) error
GetName() string GetName() string
} }
+14 -3
View File
@@ -11,11 +11,22 @@ func StdOutput() *stdoutput {
return &stdoutput{} return &stdoutput{}
} }
func (so *stdoutput) Write(str string, images []Image) error { func (so *stdoutput) Write(post Post) (Ref, error) {
fmt.Println(str) if post.ReplyTo != nil {
for _, img := range images { fmt.Printf("[reply to: %s]\n", post.ReplyTo.ID)
}
fmt.Println(post.Text)
for _, img := range post.Images {
fmt.Printf("[image: %s (%s, %d bytes)]\n", img.Filename, img.ContentType, len(img.Data)) fmt.Printf("[image: %s (%s, %d bytes)]\n", img.Filename, img.ContentType, len(img.Data))
} }
if post.Preview != nil {
fmt.Printf("[preview: %s - %s (%s)]\n", post.Preview.Title, post.Preview.Description, post.Preview.URL)
}
return Ref{ID: post.Text}, nil
}
func (so *stdoutput) Delete(ref Ref) error {
fmt.Printf("[deleted: %s]\n", ref.ID)
return nil return nil
} }
+28 -6
View File
@@ -3,6 +3,7 @@ package output
import ( import (
"bytes" "bytes"
"context" "context"
"errors"
"fmt" "fmt"
"os" "os"
@@ -44,26 +45,47 @@ func (to *twitteroutput) newOAuth1Client(accessToken, accessSecret string) error
return err return err
} }
func (to *twitteroutput) Write(str string, images []Image) error { func (to *twitteroutput) Write(post Post) (Ref, error) {
mediaIDs := make([]string, 0, len(images)) mediaIDs := make([]string, 0, len(post.Images))
for _, img := range images { for _, img := range post.Images {
mediaID, err := to.uploadImage(img) mediaID, err := to.uploadImage(img)
if err != nil { if err != nil {
return fmt.Errorf("uploading %s: %w", img.Filename, err) return Ref{}, fmt.Errorf("uploading %s: %w", img.Filename, err)
} }
mediaIDs = append(mediaIDs, mediaID) mediaIDs = append(mediaIDs, mediaID)
} }
p := &types.CreateInput{ p := &types.CreateInput{
Text: gotwi.String(str), Text: gotwi.String(post.Text),
} }
if len(mediaIDs) > 0 { if len(mediaIDs) > 0 {
p.Media = &types.CreateInputMedia{ p.Media = &types.CreateInputMedia{
MediaIDs: mediaIDs, MediaIDs: mediaIDs,
} }
} }
// A preview needs no special handling here: Twitter builds its own card
// from the URL that is already in the text.
if post.ReplyTo != nil && post.ReplyTo.ID != "" {
p.Reply = &types.CreateInputReply{
InReplyToTweetID: post.ReplyTo.ID,
}
}
_, err := managetweet.Create(context.Background(), to.client, p) out, err := managetweet.Create(context.Background(), to.client, p)
if err != nil {
return Ref{}, err
}
if out.Data.ID == nil {
return Ref{}, errors.New("created tweet has no ID")
}
return Ref{ID: *out.Data.ID}, nil
}
func (to *twitteroutput) Delete(ref Ref) error {
if ref.ID == "" {
return errors.New("no tweet ID to delete")
}
_, err := managetweet.Delete(context.Background(), to.client, &types.DeleteInput{ID: ref.ID})
return err return err
} }
+74
View File
@@ -0,0 +1,74 @@
package main
import (
"encoding/json"
"fmt"
"net/url"
"strings"
"tweetdistributor/output"
)
// findYouTubeURL returns the first YouTube video link in content, or "" if
// there is none.
func findYouTubeURL(content string) string {
for _, match := range urlPattern.FindAllString(content, -1) {
raw := trimURL(match)
u, err := url.Parse(raw)
if err != nil {
continue
}
switch strings.ToLower(u.Hostname()) {
case "youtu.be":
if strings.Trim(u.Path, "/") != "" {
return raw
}
case "youtube.com", "www.youtube.com", "m.youtube.com", "music.youtube.com":
if u.Path == "/watch" && u.Query().Get("v") != "" {
return raw
}
if strings.HasPrefix(u.Path, "/shorts/") || strings.HasPrefix(u.Path, "/live/") {
return raw
}
}
}
return ""
}
// oEmbedResponse is the part of YouTube's oEmbed document we care about.
type oEmbedResponse struct {
Title string `json:"title"`
AuthorName string `json:"author_name"`
ThumbnailURL string `json:"thumbnail_url"`
}
// youtubePreview builds the preview card for a YouTube link by asking
// YouTube's oEmbed endpoint for the title, channel and thumbnail.
func youtubePreview(videoURL string) (*output.Preview, error) {
endpoint := "https://www.youtube.com/oembed?format=json&url=" + url.QueryEscape(videoURL)
body, err := fetch(endpoint)
if err != nil {
return nil, fmt.Errorf("fetching preview for %s: %w", videoURL, err)
}
var oembed oEmbedResponse
if err := json.Unmarshal(body, &oembed); err != nil {
return nil, fmt.Errorf("parsing preview for %s: %w", videoURL, err)
}
preview := &output.Preview{
URL: videoURL,
Title: oembed.Title,
Description: oembed.AuthorName,
}
if oembed.ThumbnailURL != "" {
thumb, err := downloadImage(oembed.ThumbnailURL, "thumbnail.jpg", "image/jpeg")
if err != nil {
// A card without its thumbnail is still worth posting.
return preview, nil
}
preview.Thumb = &thumb
}
return preview, nil
}
+33
View File
@@ -0,0 +1,33 @@
package main
import "testing"
func TestFindYouTubeURL(t *testing.T) {
tests := []struct {
name string
content string
want string
}{
{"watch", "みてみて https://www.youtube.com/watch?v=dQw4w9WgXcQ おもしろい", "https://www.youtube.com/watch?v=dQw4w9WgXcQ"},
{"short host", "https://youtu.be/dQw4w9WgXcQ?t=42", "https://youtu.be/dQw4w9WgXcQ?t=42"},
{"shorts", "https://www.youtube.com/shorts/abc_123", "https://www.youtube.com/shorts/abc_123"},
{"live", "https://youtube.com/live/abc-123", "https://youtube.com/live/abc-123"},
{"mobile", "https://m.youtube.com/watch?v=abc&feature=share", "https://m.youtube.com/watch?v=abc&feature=share"},
{"trailing punctuation", "これ→https://youtu.be/abc123。", "https://youtu.be/abc123"},
{"first of several", "https://youtu.be/one https://youtu.be/two", "https://youtu.be/one"},
{"skips other links", "https://example.com/watch?v=x https://youtu.be/abc", "https://youtu.be/abc"},
{"no url", "ただのつぶやき", ""},
{"other site", "https://example.com/", ""},
{"youtube without video", "https://www.youtube.com/", ""},
{"watch without v", "https://www.youtube.com/watch?list=PL123", ""},
{"lookalike host", "https://youtube.com.evil.example/watch?v=abc", ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := findYouTubeURL(tt.content); got != tt.want {
t.Errorf("findYouTubeURL(%q) = %q, want %q", tt.content, got, tt.want)
}
})
}
}
+125
View File
@@ -0,0 +1,125 @@
// Package store remembers which post each output produced for a given Discord
// message, so a later delete or reply on Discord can be mirrored.
package store
import (
"encoding/json"
"fmt"
"os"
"sync"
"tweetdistributor/output"
)
// maxEntries bounds how far back a delete or reply can reach; the oldest
// message is forgotten once the limit is passed.
const maxEntries = 1000
// Refs maps an output name to the post that output published.
type Refs map[string]output.Ref
type Store struct {
mu sync.Mutex
path string
refs map[string]Refs
order []string
}
// New returns a store backed by the JSON file at path, loading it if it
// already exists. An empty path keeps everything in memory only, in which
// case the mapping is lost on restart.
func New(path string) (*Store, error) {
s := &Store{
path: path,
refs: map[string]Refs{},
}
if path == "" {
return s, nil
}
data, err := os.ReadFile(path)
if os.IsNotExist(err) {
return s, nil
}
if err != nil {
return nil, fmt.Errorf("reading %s: %w", path, err)
}
var saved struct {
Order []string `json:"order"`
Refs map[string]Refs `json:"refs"`
}
if err := json.Unmarshal(data, &saved); err != nil {
return nil, fmt.Errorf("parsing %s: %w", path, err)
}
if saved.Refs != nil {
s.refs = saved.Refs
}
s.order = saved.Order
return s, nil
}
func (s *Store) Put(messageID string, refs Refs) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, exists := s.refs[messageID]; !exists {
s.order = append(s.order, messageID)
}
s.refs[messageID] = refs
for len(s.order) > maxEntries {
delete(s.refs, s.order[0])
s.order = s.order[1:]
}
return s.save()
}
func (s *Store) Get(messageID string) (Refs, bool) {
s.mu.Lock()
defer s.mu.Unlock()
refs, ok := s.refs[messageID]
return refs, ok
}
func (s *Store) Delete(messageID string) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.refs[messageID]; !ok {
return nil
}
delete(s.refs, messageID)
for i, id := range s.order {
if id == messageID {
s.order = append(s.order[:i], s.order[i+1:]...)
break
}
}
return s.save()
}
// save writes the whole store out; the caller must hold the lock.
func (s *Store) save() error {
if s.path == "" {
return nil
}
data, err := json.Marshal(struct {
Order []string `json:"order"`
Refs map[string]Refs `json:"refs"`
}{Order: s.order, Refs: s.refs})
if err != nil {
return err
}
tmp := s.path + ".tmp"
if err := os.WriteFile(tmp, data, 0o600); err != nil {
return fmt.Errorf("writing %s: %w", tmp, err)
}
if err := os.Rename(tmp, s.path); err != nil {
return fmt.Errorf("replacing %s: %w", s.path, err)
}
return nil
}
+77
View File
@@ -0,0 +1,77 @@
package store
import (
"path/filepath"
"testing"
"tweetdistributor/output"
)
func TestStoreRoundTripThroughFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "posts.json")
s, err := New(path)
if err != nil {
t.Fatal(err)
}
refs := Refs{
"twitter": {ID: "123"},
"bluesky": {URI: "at://did:plc:xyz/app.bsky.feed.post/abc", CID: "bafy"},
}
if err := s.Put("discord-1", refs); err != nil {
t.Fatal(err)
}
reopened, err := New(path)
if err != nil {
t.Fatal(err)
}
got, ok := reopened.Get("discord-1")
if !ok {
t.Fatal("stored message was not found after reopening")
}
if got["twitter"] != (output.Ref{ID: "123"}) {
t.Errorf("twitter ref = %+v, want ID 123", got["twitter"])
}
if got["bluesky"].URI != refs["bluesky"].URI {
t.Errorf("bluesky URI = %q, want %q", got["bluesky"].URI, refs["bluesky"].URI)
}
if err := reopened.Delete("discord-1"); err != nil {
t.Fatal(err)
}
if _, ok := reopened.Get("discord-1"); ok {
t.Error("message is still present after Delete")
}
}
func TestStoreMissingMessage(t *testing.T) {
s, err := New("")
if err != nil {
t.Fatal(err)
}
if _, ok := s.Get("nope"); ok {
t.Error("Get returned a ref for a message that was never stored")
}
if err := s.Delete("nope"); err != nil {
t.Errorf("deleting an unknown message returned %s", err)
}
}
func TestStoreEvictsOldest(t *testing.T) {
s, err := New("")
if err != nil {
t.Fatal(err)
}
for i := 0; i < maxEntries+10; i++ {
if err := s.Put(string(rune('a'+i%26))+string(rune(i)), Refs{"stdout": {ID: "x"}}); err != nil {
t.Fatal(err)
}
}
if len(s.refs) > maxEntries {
t.Errorf("store holds %d entries, want at most %d", len(s.refs), maxEntries)
}
if len(s.order) != len(s.refs) {
t.Errorf("order has %d entries but refs has %d", len(s.order), len(s.refs))
}
}