This might be an impossible ask, I don’t know. But sometimes in a sprawling thread one exchange comes across as particularly interesting. Or more importantly, I often want to review an exchange of my own before writing the next reply. Currently, this exchange might be buried in a sea of cross talk. It would be great if we could filter a thread so that only messages that involve the specified users, through quoting, replying, or @ing. This way specific exchanges can be easily followed.
Is anyone else interested in this? Is it possible?
That’s the beauty of Discourse and open source, m8. Anyone can make a plugin, and the code is free to look at to understand how things like the frontend work. Assuming that this plugin doesn’t exist, anyone here who knows how to program can probably make one, possibly with difficulty maybe? IDK since I’m not a programmer, but it sounds possible in principle. Take for example the raw forum text of The Grounding of Logic thread here. It looks like like there’s at least some things we can extract to keep track of a reply chain, such as the quote function.
I have always wanted that as well, and have thought about writing the code for it.
You can use the advanced search as a workaround. For example, if you want to return every post of hypericin’s that contains the word “Mijin1” within topic 993, sorted reverse chronologically, you can use <this search>. This gives you a list of all the posts you wrote to Mijin1 in that thread.
Of course one can also traverse a thread backwards by clicking the up-arrow in each quote block, or they can traverse a thread forwards by using the Replies Dropdown at the bottom of each post.* This allows you to navigate to the posts that immediately preceded or succeeded a post within a dialogue.
Beyond that, you can click the “expand” symbol in each quote block to see the entirety of the post being quoted, and you can do this recursively to open the entire conversation within a single post.
* Obviously only posts which have been replied to will have a Replies Dropdown
At the risk of angering the powers that be, I wonder if you could just ask AI.
From what I’ve been made aware, ability to utilize user-made plug-ins requires the next payment tier up from the current one, which if I recall is like five times as much, like 500 USD a month or so. Hey, if you got the scratch, pony up and I’m sure ol’ what’s his name will sign us up.
Short of that, the advanced search operators seem reasonable. But, if I could offer a humble analog-ish approach. If a discussion is really that interesting, just right click on your desktop → New Text Document and copy and past the exchanges separated by a series of dashes (- - -) or something. Even a dozen replies should take less than a minute if you’re of good health. Then you can gaze upon said exchanges at your leisure and to your heart’s content—with or without Internet, I might add. You can then even print them out and carry them with you wherever you may go.
Good idea. It would be nice if it was built into the search. You can request features on the Discourse Meta forum. The list of feature requests is at Feature - Discourse Meta.
Apart from that, what I would do is Ctrl+P, save as PDF, then ask Claude or ChatGPT to extract the posts you want and give you back a document with just those posts.
Within TPF you can use the built-in topic navigation tools to trace a conversation via replies, but that might not include quotes and mentions.
I’m finding this quite clumsy, even after having used it for a while. In the old forum it was relatively easy to have multiple windows open with various posts, but the way that the new site works this means multiple editing windows that tend to get muddled.
I’m thinking the easiest way for me would be to write a script that takes a topic url, saves as pdf, and trims the pdf so it only includes the relevant messages.
(Didn’t mean to reply to you Banno. Which brings up another new nuisance. Once a certain message is replied to, that reply cannot be edited out).
I wrote a small python script that takes a post id (the first number in the url after the title), and two users, and outputs html and markdown text which only contains posts involving both users.
You will need to be familiar enough with python to run a script and install dependencies. Let me know if you get any use out of it, and if there are issues.
#Discourse discussion filter by hypericin
#v1.0
#filters a discourse topic so only posts involving two specified users are displayed
#usage: philo.py <topic_id> <user1> <user2>
#outputs html and markdown text in the current directory
import os
import argparse
import re
import json
import requests
import html2text
#parse arguments: topic_id, user1, user2
argsParser = argparse.ArgumentParser()
argsParser.add_argument("topic_id", type=int)
argsParser.add_argument("user1", type=str)
argsParser.add_argument("user2", type=str)
args = argsParser.parse_args()
topic_id = args.topic_id
users = (args.user1.lower(), args.user2.lower())
url = f"https://www.thephilosophyforum.com/t/"
#retrieve topic data, and inital batch of posts
s = requests.Session()
s.headers.update({"Accept": "application/json", "User-Agent": "Mozilla/5.0"})
topic = s.get(f"{url}{topic_id}.json")
topic.raise_for_status()
j = topic.json()
posts = j["post_stream"]["posts"]
#Discourse streams posts, only 20 or so are sent on the first batch.
#figure out which posts are missing.
seen = {p["id"] for p in posts}
missing_ids = [
pid for pid in j["post_stream"]["stream"]
if pid not in seen
]
#Retrieve remaining posts, if any, using the streaming endpoint
for i in range(0, len(missing_ids), 50):
params = [("post_ids[]", pid) for pid in missing_ids[i:i+50]]
r = s.get(f"{url}{topic_id}/posts.json", params=params)
r.raise_for_status()
newPosts = r.json()["post_stream"]["posts"]
posts.extend(newPosts)
#exit early if no posts found
if len(posts) < 1:
print("Error: no posts retrieved")
exit()
#initialize output and html
title = f"{j['title']} ({args.user1}, {args.user2})"
html = f"<!DOCTYPE html>\n<title>{title}</title>\n"
#retrieve all post ids involving both specified participants
goodPosts = []
for post in posts:
postIdx = post["post_number"] - 1
userCase = post["username"]
user = userCase.lower()
#exit if post is by neither user
if user != users[0] and user != users[1]:
continue
otherUser = users[0 if user == users[1] else 1]
#add this post if other user is mentioned
if re.search(f"@{otherUser}", post["cooked"], re.IGNORECASE):
goodPosts.append(postIdx)
#search through all quotes. if the other user is quoted, add both the current and quoted posts
pattern = (
r'data-username="([^"]+)"\s+'
r'data-post="([^"]+)"\s+'
r'data-topic="([^"]+)"'
)
quotes = re.findall(pattern, post["cooked"])
for q in quotes:
if q[0].lower() == otherUser:
goodPosts.append(postIdx)
#only add the quoted post if it belongs to this topic
if int(q[2]) == j["id"]:
goodPosts.append(int(q[1])-1)
#if the other user is replied to, and this and replied to post
#If the reply is to the other user's OP, and the user is quoting someone else, ignore
replyIdx = post["reply_to_post_number"]
replyIdx = 0 if replyIdx is None else replyIdx - 1
if (posts[replyIdx]["username"].lower() == otherUser) and (len(quotes) == 0 or replyIdx > 0):
goodPosts.append(postIdx)
goodPosts.append(replyIdx)
#sort and make unique the relevant post ids
goodPosts = sorted(list(set(goodPosts)))
if len(goodPosts) == 0:
print("Error: no posts match users")
exit()
print(goodPosts)
#build html output
for p in goodPosts:
html += f"<hr><h3>{posts[p]["username"]}</h3>{posts[p]["cooked"]}"
#remove forbidden charactrers from title, and set path to cwd
filePath = title.replace(' ', '_')
filePath = filePath.translate(str.maketrans("", "", '<>?:"/\\*|'))
filePath = f"{os.getcwd()}/{filePath}"
#write html
with open(filePath + ".html", "w", encoding="utf-8") as file:
file.write(html)
print ("wrote " + filePath + ".html");
#convert html to markdown
markdown = html2text.html2text(html)
#get rid of user icon links
markdown = re.sub(r'!\[\]\([^)]*\)', '', markdown)
#write
with open(filePath + ".txt", "w", encoding="utf-8") as file:
file.write(markdown)
print ("wrote " + filePath + ".txt");
Good work; I may use it. It would make a nice theme component for Discourse (we wouldn’t be able to use a custom plugin on TPF, but we can add theme components, which are fully client-side).
I made something similar for my own purposes as a Chrome extension. If this is not prohibited by the forum rules, I can share the code and instructions.
These days, on many websites, you can quite often get banned for posting a link to a GitHub project, as it may be treated as unpaid advertising, potentially dangerous software, or something else. If that is not prohibited here, I can upload it to GitHub and share the link.
By modern standards, this is a very simple project, created with ChatGPT Codex in about two hours or so.
Not me personally, as I am not a programmer. (I wonder if there are any programmers on this site?)[1] But it’s certainly possible. However, there would be some hurdles to this. @hypericin points out that
Relying on quotations in the raw text can therefore be unreliable since there’s no way to syntactically follow reply chains. But to get over this hurdle, maybe someone can use AI to keep track of reply chains semantically.
My only point being: it’s possible given the technical openness of Discourse. Anyone can make a plugin[2] and @Jamal can add it to the site if they want.[3]
The more difficult hurdle, though, would be to maintain that plugin over time.
EDIT: I just caught up with the thread. It looks like there are, and they have developed their own way of creating this feature. ↩︎
