ComputerCraft Forums

ComputerCraft => Programs => Topic started by: osmarks on Aug 13, 2018, 10:33 AM

Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: osmarks on Aug 13, 2018, 10:33 AM
As should be known to any player on Switchcraft, going through lots of chests manually to find items is annoying. So Turtlegistics was made. However, it doesn't deal well with having multiple Turtlegisticses running on the same set of chests, and inefficiently fetches the display name of every item, making it bad for larger chest setups. There are modified versions fixing the latter, but not the former.

I decided to create a new system, from the ground up, to handle large chest arrays and multiple clients better. That was called Dragon, and had somewhat broken networking/error handling, so I rewrote most of it again and copied a bit of code over, into Wyvern.

Wyvern aims to:

Currently only a CLI client, automated IO interface, and backend for chest storage are available. This is enough for most setups, however.

Read more - and get installation instructions, etc - here. (https://osmarks.tk/git/osmarks/wyvern/src/branch/master/README.md)

It also has an optional acronym: "Working Yet Veritably Excellent Retrieval Network".
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: HydroNitrogen on Aug 13, 2018, 06:04 PM
Is it wireless? If so, I think it would be great if you'd make it wireless! (This concept is possible using a bound introspection module in a manipulator)

Currently I do not yet see the benefits of a CLI based Wyvern over GUI based SquidDev's Artist (https://github.com/SquidDev-CC/artist), which is very fast, does great searching and has a convenient GUI.

But perhaps wireless options may change that!
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: osmarks on Aug 13, 2018, 06:30 PM
Quote from: HydroNitrogen on Aug 13, 2018, 06:04 PMIs it wireless? If so, I think it would be great if you'd make it wireless! (This concept is possible using a bound introspection module in a manipulator)

Currently I do not yet see the benefits of a CLI based Wyvern over GUI based SquidDev's Artist (https://github.com/SquidDev-CC/artist), which is very fast, does great searching and has a convenient GUI.

But perhaps wireless options may change that!
It's designed for large setups, does not require an introspection module, has an API for automation (documentation coming Soon(tm)) and allows multiple terminals with synchronised data.

Wireless/introspection module access will come when I can be bothered to make the overlay glasses client, which will have the benefit of being able to display item icons. This will probably be around when SC updates to whichever version of plethora allows 3D overlay glass objects.

Also, searching is currently done via use of Levenshtein distance to check how close your search term is to the display name of items. I am looking into other approaches but haven't found any good ones, though this has its problems too ("wool" pulls not red wool but bows and coal due to... stuff).
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: Incin on Aug 13, 2018, 06:52 PM
Quote from: osmarks on Aug 13, 2018, 06:30 PMAlso, searching is currently done via use of Levenshtein distance
- snip -
though this has its problems too ("wool" pulls not red wool but bows and coal due to... stuff).

The problem is that it is easier to transform wool to coal than to red wool because to get to red wool you have to insert 4 characters, vs the 2 modifications to transform to coal.

To mitigate this, you should match against all (consecutive) substrings of length n of the strings you match against, where n is the length of your search string. If you need more information, checkout the search functionality used for the luadash documentation utility (https://github.com/tmpim/luadash/blob/master/luadash.lua#L75).
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: SquidDev on Aug 13, 2018, 07:04 PM
Quote from: Incin on Aug 13, 2018, 06:52 PMThe problem is that it is easier to transform wool to coal than to red wool because to get to red wool you have to insert 4 characters, vs the 2 modifications to transform to coal.
Well, the bigger issue is that Levenshtein distance isn't a great metric for fuzzy matching. If I enter "cble", one'd expect to get "cobblestone". However, "chest" is actually closer!

I believe most fuzzy systems (and at least BSRocks' and Artist's) try to match up pattern letters against input letters in order and then score based on a series of factors like number of consecutive matching letters. You miss out on some benefits of Levenshtein (such as detecting transpositions), but it's not a great loss.
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: osmarks on Aug 13, 2018, 07:20 PM
Quote from: SquidDev on Aug 13, 2018, 07:04 PM
Quote from: Incin on Aug 13, 2018, 06:52 PMThe problem is that it is easier to transform wool to coal than to red wool because to get to red wool you have to insert 4 characters, vs the 2 modifications to transform to coal.
Well, the bigger issue is that Levenshtein distance isn't a great metric for fuzzy matching. If I enter "cble", one'd expect to get "cobblestone". However, "chest" is actually closer!

I believe most fuzzy systems (and at least BSRocks' and Artist's) try to match up pattern letters against input letters in order and then score based on a series of factors like number of consecutive matching letters. You miss out on some benefits of Levenshtein (such as detecting transpositions), but it's not a great loss.
If you have a link to some nice implementations of fuzzy search (mine is stolen from luadash) that I could use instead, please give me them.
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: SquidDev on Aug 13, 2018, 07:34 PM
Quote from: osmarks on Aug 13, 2018, 07:20 PMIf you have a link to some nice implementations of fuzzy search (mine is stolen from luadash) that I could use instead, please give me them.
This is Artist's (https://github.com/SquidDev-CC/artist/blob/vnext/artist/lib/match.lua) implementation. It's incredibly simple (well, compared with the previous one (https://github.com/SquidDev-CC/artist/blob/master/artist/gui/match.lua)) but seems to do a good enough job for most cases. There's some definite improvements you can make, such as treating _, camelCase and whitespace as equivalent, handling transpositions, etc... but it seems to work.
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: osmarks on Aug 13, 2018, 08:42 PM
Quote from: SquidDev on Aug 13, 2018, 07:34 PM
Quote from: osmarks on Aug 13, 2018, 07:20 PMIf you have a link to some nice implementations of fuzzy search (mine is stolen from luadash) that I could use instead, please give me them.
This is Artist's (https://github.com/SquidDev-CC/artist/blob/vnext/artist/lib/match.lua) implementation. It's incredibly simple (well, compared with the previous one (https://github.com/SquidDev-CC/artist/blob/master/artist/gui/match.lua)) but seems to do a good enough job for most cases. There's some definite improvements you can make, such as treating _, camelCase and whitespace as equivalent, handling transpositions, etc... but it seems to work.
What are the return values there?
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: osmarks on Aug 14, 2018, 09:35 PM
Fancier fuzzy search has been implemented, though I may remove it if it doesn't work significantly better than the old one.

Coming Very Soon(tm), mostly because I need it right now: an automated IO (combination ME interface/import bus/export bus).

EDIT: The above is done!
I also implemented non-fuzzy search (prefix your query with !).
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: QuickMuffin8782 on Aug 10, 2020, 11:31 AM
Bro, my anti-malware (OpenDNS) had this, and ALSO blocked copy-cat as well... (it's on my iPad...)
This site is blocked due to a security threat that was discovered by the Cisco Umbrella security researchers.

ACType: 0
Block Type: security
Bundle ID: 912155
Domain Tagging: -
Host: malware.opendns.com
IP Address: 162.210.247.201
Org ID: 2310278
Origin ID: 131701314
Prefs: -
Query: url=osmarks.tk%2Fgit%2Fosmarks%2Fwyvern%2Fsrc%2Fbranch%2Fmaster%2FREADME.md&server=dfw15&prefs=&tagging=&nref
Server: dfw15
Title: Wyvern - Next-Generation Centralized Plethora Storage
Post by: QuickMuffin8782 on Aug 11, 2020, 07:22 PM
Why not post it to pastebin (https://pastebin.com/) for the program?
It will be useful, so people who cannot get to github websites due to the Cisco Umbrella Security Developers who find that github's pages have a security problem (probably because of a update).
It's free to create a account to have your pastes (programs) stuff, also...
I also have a account (https://pastebin.com/u/quickmuffin8783) as well, you can check out my pastes (https://pastebin.com/u/quickmuffin8783) as well.