37 Commits

Author SHA1 Message Date
  Dryusdan fa1550e26a Merge branch 'improve-randomness' of framasky/masto-image-bot into master 1 year ago
  Luc Didry e5e4938479
Improve local images randomness by using random.SystemRandom() 1 year ago
  Dryusdan 67849b7f9f Remove quote for unsplash_client_id 1 year ago
  Tristan Le Chanony 38c1afa81a Improve README 2 years ago
  Tristan Le Chanony d7287622b9 Improve unsplash mode 2 years ago
  Dryusdan 1a8f9a381a Merge branch 'add-unsplash-random-source' of framasky/masto-image-bot into master 2 years ago
  Dryusdan 995370b7ea Merge branch 'update-systemd-service' of framasky/masto-image-bot into master 2 years ago
  Luc Didry cc3cbd2fe7
Add unsplash-random source 2 years ago
  Luc Didry 8aef6ae903
Update systemd documentation 2 years ago
  Dryusdan 0063745171 Merge branch 'linting' of framasky/masto-image-bot into master 2 years ago
  Dryusdan 8332357d9a Merge branch 'improve-readme' of framasky/masto-image-bot into master 2 years ago
  Luc Didry 43acb49171
Improve README (translation, install deps…) 2 years ago
  Luc Didry 97b2c6b01e
Lintings changes 2 years ago
  Dryusdan 320c7056d0 Mettre à jour 'bot.py' 2 years ago
  Dryusdan d9ef0dc3af Resize image (because mastodon not accept big picture) 2 years ago
  Dryusdan cf905aac3c add renamer 2 years ago
  Dryusdan fde735c88a correct respond in thread #10 2 years ago
  Dryusdan 975001bd4f Remove uselessfile 2 years ago
  Dryusdan 7261eacccd improve doc 2 years ago
  Dryusdan a5633e5c18 select image recursive folder #4 2 years ago
  Dryusdan 0b27b154b7 add research on multiple folder 2 years ago
  Dryusdan 1c46723cd3 improve doc 2 years ago
  Dryusdan 15458cfb2c improve doc 2 years ago
  Dryusdan 8e7f69ecc7 add distant image #3 2 years ago
  Dryusdan 8fc886351f make blacklist #8 2 years ago
  Dryusdan 3d28bcc9de check if account is bot #9 2 years ago
  Dryusdan ea6f6e9fa3 add hour limiter #7 2 years ago
  Dryusdan cb15c9a4b4 add limiter #7 2 years ago
  Dryusdan 69355227b6 correction of mask image on a toot 2 years ago
  Dryusdan 012d6f9f98 improve doc 2 years ago
  Dryusdan 1e35632ede add limiter 2 years ago
  Dryusdan 2585b75320 improve doc 2 years ago
  Dryusdan 7896a368b0 correct always hide picture 2 years ago
  Dryusdan ab94b056b9 Improve readme 2 years ago
  Dryusdan 7adf21fe65 replace wrong parameter by good (logic) 2 years ago
  Dryusdan 0816a66da3 Configuration in config file 2 years ago
  Dryusdan a7da455862 use config file 2 years ago
11 changed files with 339 additions and 111 deletions
Split View
  1. +3
    -0
      .gitignore
  2. +53
    -11
      README.md
  3. +0
    -20
      TootHTMLParser.py
  4. +4
    -0
      blacklist.sample.json
  5. +237
    -79
      bot.py
  6. +1
    -0
      collection.sample.json
  7. +12
    -1
      config.sample.txt
  8. +0
    -0
      limiter/hour/README.md
  9. +0
    -0
      limiter/minute/README.md
  10. +28
    -0
      rename_all_file.sh
  11. +1
    -0
      requirements.txt

+ 3
- 0
.gitignore View File

@ -3,3 +3,6 @@ secrets/secrets.txt
config.txt
__pycache__/*
TootHTMLParser.pyc
blacklist.json
collection.json
output.jpg

+ 53
- 11
README.md View File

@ -1,34 +1,66 @@
# masto-image-bot
Un bot qui récupère une image random en local et la publie
A bot that fetches a random local image and publish it on Mastodon.
Copiez le fichier `config.sample.txt` en `config.txt`, ajoutez le chemin de votre dossier image.
Remplissez le fichier `secrets/secrets.txt` et remplissez le avec les code que vous trouverez dans l'onglet développeur de votre compte Mastodon.
Copy the file `config.sample.txt` to `config.txt` and add the path to your images directory.
File the file `secrets/secrets.txt` with the codes you will find in the developper tab of your Mastodon account.
You can also register your bot on a Mastodon instance and get the needed codes with the help of the script [register-app](https://framagit.org/fiat-tux/hat-softwares/mastodon/register-app).
## Configure it
## Install the dependencies
replace `bot_name` variable in `bot.py` with name of your bot without domain
```
pip3 install -r requirements.txt
```
## Configure the bot
Copy `config.sample.txt` to `config.txt` and replace data by your data.
If you don't want any "spoiler text", just leave the line empty.
| **Field name** | Description | Value |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------- | ---------------------- |
| img_path | Path where image are located | /home/bot/img | ../bot |
| name | Name of your bot (name after @). Is usefull to hide it's name in reply | mybot |
| secrets_filepath | Path where located secret | secrets/secrets.txt |
| log_filepath | Path where located log file | activity.log |
| blacklist_filepath | Path where located blacklist file | blacklist.json |
| collection_filepath | Path where located distant image collection | collection.json |
| sensitive | Hide picture behind "sensitive content" mask or not | yes | no |
| default_text | Text for --img option | string |
| spoiler_text | Text for every spoiler (or CW) (for --img or --stream option) | string |
| limit | Limit send per minute per person | int |
| limit_hour | Limit send par hour per person | int |
| collection_url | URL of website you deserve image. `<collection>` is a variable who depend on collection.json (you can remove this variable) | string |
| unsplash_client_id | Access key of your Unsplash App (you can create it on api.unsplash.com ) | string |
Copy `blacklist.sample.json` to `blacklist.json` and replace or add accounts that should not receive any image
Copy `collection.sample.json` to `collection.json` and add collection for your bot
## Use it
```
usage: bot.py [-h] [-i] [-s]
usage: bot.py [-h] [-i] [-s SOURCE] [--stream]
Choose between image or streaming
optional arguments:
-h, --help show this help message and exit
-i, --img post image
-s, --stream stream user profile
-h, --help show this help message and exit
-i, --img post image
-s SOURCE, --source SOURCE
Source of image [ local | distant | unsplash-random ]
--stream stream user profile
```
`--img` option send image.
`--stream` option send image on mention
## Create a systemd service
On `/etc/systemd/system/bot.service` copy paste code behind
```
[Unit]
Description=Image bot Mastodon
@ -41,5 +73,15 @@ Type=simple
User=masto-bot
TimeoutSec=15
WorkingDirectory=/home/masto-bot/
ExecStart=/usr/bin/python3 bot.py --stream
ExecStart=/usr/bin/python3 bot.py --stream --source=local
[Install]
WantedBy=multi-user.target
```
Then do
```
systemctl daemon-reload
systemctl enable bot.service
systemctl start bot.service
```

+ 0
- 20
TootHTMLParser.py View File

@ -1,20 +0,0 @@
from html.parser import HTMLParser
class TootHTMLParser(HTMLParser):
def __init__(self):
super().__init__()
self.txt = ""
def handle_data(self, data):
self.txt += str(data).lstrip().rstrip().lower() + " "
#
#
# content = ""
# with open("input") as f:
# content = f.readlines()
# content = set(content)
# parser = TootHTMLParser()
# for word in content:
# parser.feed(word)
# with open("output", "w+") as f:
# f.write(parser.txt)

+ 4
- 0
blacklist.sample.json View File

@ -0,0 +1,4 @@
[
"user@domain.tld",
"userInYouInstance"
]

+ 237
- 79
bot.py View File

@ -1,116 +1,274 @@
#!/usr/bin/env python3
# coding: utf-8
# -*- coding: utf-8 -*-
# A Fediverse (decentralized social network, for instance using Mastodon) bot
from mastodon import StreamListener
from lxml import html
from logging.handlers import RotatingFileHandler
from pprint import pprint
from random import randint
from utils.config import get_parameter, init_log, init_mastodon
from PIL import Image
from io import BytesIO
import requests, os, random, sys, time, json, logging, argparse, re, shutil
config_file = "config.txt"
secrets_filepath = get_parameter("secrets_filepath", config_file)
log_filepath = get_parameter("log_filepath", config_file)
blacklist_filepath = get_parameter("blacklist_filepath", config_file)
collection_filepath = get_parameter("collection_filepath", config_file)
log = init_log(log_filepath)
mastodon = init_mastodon(config_file, secrets_filepath)
import requests, os, random, sys, time, json, logging, argparse, re
blacklist_file = open(blacklist_filepath,'r')
BLACKLIST = json.loads(blacklist_file.read())
blacklist_file.close()
config_file = "config.txt"
secrets_filepath = "secrets/secrets.txt"
log_filepath = "activity.log"
log = init_log(log_filepath)
bot_name = "nosafe"
mastodon = init_mastodon(config_file, secrets_filepath)
mime_dict = {'.jpg': 'image/jpeg', '.jpe': 'image/jpeg', '.jpeg': 'image/jpeg', '.png': 'image/png', '.gif': 'image/gif'}
def post_img(mastodon, text, visibility, log, config):
def post_img_local(mastodon, text, log, config):
img_path = get_parameter("img_path", config)
file = random.choice(os.listdir(img_path+"/"))
image_byte = open(img_path+"/"+file, "rb").read()
file, ext = os.path.splitext(file)
mime_dict = {'.jpg': 'image/jpeg', '.jpe': 'image/jpeg', '.jpeg': 'image/jpeg', '.png': 'image/png', '.gif': 'image/gif'}
mime = mime_dict[str.lower(ext)]
#try:
continu = True;
while continu:
secure_random = random.SystemRandom()
file = secure_random.choice(os.listdir(img_path+"/"))
if os.path.isdir(img_path+file):
img_path = img_path+file+"/"
else:
if ".zip" not in file:
continu = False
im = Image.open(img_path+ file)
width, height = im.size
NEW_WIDTH = 2048
if width > 2048:
difference_percent = NEW_WIDTH / width
new_height = height * difference_percent
size = new_height, NEW_WIDTH
im = im.resize((int(NEW_WIDTH), int(new_height)))
im.save('resize_img.jpg')
file = "resize_img.jpg"
shutil.copyfile(file, "/tmp/"+file)
else:
log.debug("no resize")
shutil.copyfile(img_path+file, "/tmp/"+file)
image_byte = open("/tmp/"+file, "rb").read()
file, ext = os.path.splitext(file)
os.remove("/tmp/"+file+ext)
#except KeyError:
# mime = None;
# pass
try:
mime = mime_dict[str.lower(ext)]
except KeyError:
mime = None;
log.error(ext + " is not present on mime_dict, please add this")
pass
media_dict = mastodon.media_post(image_byte, mime)
return media_dict;
def post_unsplash_random_image(mastodon, log, config):
collection_url = get_parameter("collection_url", config)
unsplash_client_id = get_parameter("unsplash_client_id", config)
collecion_file = open(collection_filepath,'r')
collections = json.loads(collecion_file.read())
collecion_file.close()
count_collection = len(collections)-1
if count_collection > -1:
id_collection = randint(0,count_collection)
collection_url="&collections="+str(collections[id_collection])
else:
collection_url=''
response = requests.get("https://api.unsplash.com/photos/random?client_id="+unsplash_client_id+collection_url)
randim_json = json.loads(response.text)
randim_url = "{}&q=85&crop=entropy&cs=tinysrgb&w=2048&fit=max".format(randim_json['urls']['raw'])
img_response = requests.get(randim_url)
pattern = Image.open(BytesIO(img_response.content), "r").convert('RGB')
pattern.save('output.jpg')
media_dict = mastodon.media_post("output.jpg")
toot = "Shot by {} ({})\n{}".format(randim_json['user']['name'], randim_json['user']['links']['html'], randim_json['links']['html'])
return { 'media_dict': media_dict, 'toot': toot };
def post_img_distant(mastodon, text, log, config):
collection_url = get_parameter("collection_url", config)
collecion_file = open(collection_filepath,'r')
collections = json.loads(collecion_file.read())
collecion_file.close()
count_collection = len(collections)-1
id_collection = randint(0,count_collection)
collection_url = collection_url.replace("<collection>", str(collections[id_collection]))
response = requests.get(collection_url)
pattern = Image.open(BytesIO(response.content), "r").convert('RGB')
pattern.save('output.jpg')
media_dict = mastodon.media_post("output.jpg")
return media_dict;
cleanr = re.compile('<.*?>')
def cleanhtml(raw_html):
cleanr = re.compile('<.*?>')
cleantext = re.sub(cleanr, '', raw_html)
return cleantext
class BotListener(StreamListener):
# use only notification
def on_notification(self, notification):
# catch only mention in notification
if notification['type'] == 'mention':
log.debug("Got a mention")
sender = notification['account']['acct'] # Get sender name
sender_filename = "limiter/" + sender; # Forge file for limiter
if os.path.isfile(sender_filename): # Check if file exist
statbuf = os.stat(sender_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
pprint(last_edit)
pprint(ts)
if ts - last_edit > 59: # check if file is modified 1 minute after last edition
f = open(sender_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
f = open(sender_filename,'r+')
number_of_mention = int(f.read())
if number_of_mention < 4: # limit of mention per minute is 4
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
can_continue = False # if number of mention is for, user can't receive anything
f.close()
else: # File not exist, create it and initialise it
f = open(sender_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
if can_continue:
id = notification['status']['id']
visibility = notification['status']['visibility']
if visibility == 'public':
visibility = 'unlisted'
mentions = notification['status']['mentions']
text = "@" + notification['status']["account"]["acct"] + " "
for mention in mentions:
if mention["acct"] != bot_name:
text = text + "@" + mention["acct"] + " "
media_dict = post_img(mastodon, "NSFW", 1, log, config_file)
mastodon.status_post(text, in_reply_to_id=id, media_ids=[media_dict], sensitive=True, visibility=visibility, spoiler_text="#NSFW")
else:
log.debug("no picture send :(")
pass
else:
log.debug("Nevermind")
def __init__(self, args):
self.args = args
# use only notification
def on_notification(self, notification):
# catch only mention in notification
if notification['type'] == 'mention':
log.debug("Got a mention")
if notification["account"]["bot"] == False:
sender = notification['account']['acct'] # Get sender name
if sender in BLACKLIST:
log.info("Service refused to %s" % sender)
return
sender_hour_filename = "limiter/hour/" + sender; # Forge file for limiter
sender_minute_filename = "limiter/minute/" + sender; # Forge file for limiter
if os.path.isfile(sender_hour_filename): # Check if file exist
log.debug("Sender file exist")
statbuf = os.stat(sender_hour_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
if ts - last_edit > 3599: # check if file is modified 1 hour after last edition
log.debug("file is too old")
f = open(sender_hour_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
log.debug("file is young")
f = open(sender_hour_filename,'r+')
limit = int(get_parameter("limit_hour", config_file))
number_of_mention = int(f.read())
if number_of_mention < limit: # limit of mention per hour is limit_hour
log.debug("Sender have less of limit requests")
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
log.debug("Sender have more of limit requests")
can_continue = False # if number of mention is for, user can't receive anything
f.close()
else: # File not exist, create it and initialise it
log.debug("Sender file not exist")
f = open(sender_hour_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
if can_continue:
if os.path.isfile(sender_minute_filename): # Check if file exist
log.debug("Sender file exist")
statbuf = os.stat(sender_minute_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
if ts - last_edit > 59: # check if file is modified 1 minute after last edition
log.debug("file is too old")
f = open(sender_minute_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
log.debug("file is young")
f = open(sender_minute_filename,'r+')
limit = int(get_parameter("limit", config_file))
number_of_mention = int(f.read())
if number_of_mention < limit: # limit of mention per minute is 4
log.debug("Sender have less of limit requests")
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
log.debug("Sender have more of limit requests")
can_continue = False # if number of mention is for, user can't receive anything
file = open(sender_hour_filename,'r+')
number_of_mention = int(file.read())
file.seek(0)
file.write(str(number_of_mention - 1))
file.close()
f.close()
else: # File not exist, create it and initialise it
log.debug("Sender file not exist")
f = open(sender_minute_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
if can_continue:
id = notification['status']['id']
visibility = notification['status']['visibility']
if visibility == 'public':
visibility = 'unlisted'
mentions = notification['status']['mentions']
text = "@" + notification['status']["account"]["acct"] + " "
for mention in mentions:
if mention["acct"] != get_parameter("name_bot", config_file):
text = text + "@" + mention["acct"] + " "
if get_parameter("sensitive", config_file) == "yes":
sensitive = True
else:
sensitive = False
if self.args.source == "local":
media_dict = post_img_local(mastodon, get_parameter("default_text", config_file), log, config_file)
elif self.args.source == "distant":
media_dict = post_img_distant(mastodon, get_parameter("default_text", config_file), log, config_file)
elif self.args.source == "unsplash-random":
resp = post_unsplash_random_image(mastodon, log, config_file)
text = text + "\n" + resp['toot']
media_dict = resp['media_dict']
mastodon.status_post(text, id, media_ids=[media_dict], sensitive=sensitive, visibility=visibility, spoiler_text=get_parameter("spoiler_text", config_file))
else:
log.debug("no picture send :(")
pass
else:
log.debug("Nevermind")
def main():
parser = argparse.ArgumentParser(description='Choose between image or streaming')
parser.add_argument("-i", "--img", action='store_true', help="post image")
parser.add_argument("-s", "--stream", action="store_true", help="stream user profile")
parser.add_argument("-s", "--source", help="Source of image [ local | distant | unsplash-random ]")
parser.add_argument("--stream", action="store_true", help="stream user profile")
args = parser.parse_args()
if args.img:
media_dict = post_img(mastodon, "NSFW", 1, log, config_file)
mastodon.status_post("", None, media_ids=[media_dict], sensitive=True, visibility='public', spoiler_text="#NSFW")
text = get_parameter("default_text", config_file)
if args.source == "local":
media_dict = post_img_local(mastodon, get_parameter("default_text", config_file), log, config_file)
elif args.source == "distant":
media_dict = post_img_distant(mastodon, get_parameter("default_text", config_file), log, config_file)
elif args.source == "unsplash-random":
resp = post_unsplash_random_image(mastodon, log, config_file)
text = resp['toot']
media_dict = resp['media_dict']
if get_parameter("sensitive", config_file) == "yes":
sensitive = True
else:
sensitive = False
mastodon.status_post(text, None, media_ids=[media_dict], sensitive=sensitive, visibility='public', spoiler_text=get_parameter("spoiler_text", config_file))
sys.exit()
elif args.stream:
stream = BotListener();
stream = BotListener(args);
while True:
try:
log.info("Start listening...")


+ 1
- 0
collection.sample.json View File

@ -0,0 +1 @@
[]

+ 12
- 1
config.sample.txt View File

@ -1,2 +1,13 @@
img_path: uri_path
name_bot: "bot"
name_bot: bot
secrets_filepath: secrets/secrets.txt
log_filepath: activity.log
blacklist_filepath: blacklist.json
collection_filepath: collection.json
sensitive: False
default_text: some text here
spoiler_text: some text here
limit: 2
limit_hour: 10
collection_url: https://source.unsplash.com/collection/<collection>/
unsplash_client_id: 03ad5bfbaa0acd6c96a728d425e533683ec25e5fb7fcf99f6461720b3d0d75a1

+ 0
- 0
limiter/hour/README.md View File


+ 0
- 0
limiter/minute/README.md View File


+ 28
- 0
rename_all_file.sh View File

@ -0,0 +1,28 @@
#!/bin/bash
function generate_random_char {
echo $( dd if=/dev/urandom bs=16 count=1|base64) > /tmp/rename_all_image
cp /tmp/rename_all_image /tmp/rename_all_image.back
sed -ie 's/[!@#\+\/$%^&*()=]//g' /tmp/rename_all_image.back
NEW_FILENAME=$(cat /tmp/rename_all_image.back)
EXTENSION=$(echo $img | cut -f 2 -d '.')
echo $NEW_FILENAME"."$EXTENSION
}
function move_file {
NEW_FILE=$(generate_random_char)
filepath=$2
IMG=$1
#echo $filepath"/"$NEW_FILE
if [ ! -f $filepath"/"$NEW_FILE ]; then
#mv $IMG $NEW_FILE
mv $IMG $filepath"/"$NEW_FILE
else
move_file $IMG
fi
}
for img in `ls $1/*`; do
filepath=$1
move_file $img $filepath
done

+ 1
- 0
requirements.txt View File

@ -1,3 +1,4 @@
Mastodon.py
lxml
requests
Pillow

Loading…
Cancel
Save