Wyvern - Next-Generation Centralized Plethora Storage

Started by osmarks, Aug 13, 2018, 10:33 am

Previous topic - Next topic

osmarks

As should be known to any player on Switchcraft, going through lots of chests manually to find items is annoying. So Turtlegistics was made. However, it doesn't deal well with having multiple Turtlegisticses running on the same set of chests, and inefficiently fetches the display name of every item, making it bad for larger chest setups. There are modified versions fixing the latter, but not the former.

I decided to create a new system, from the ground up, to handle large chest arrays and multiple clients better. That was called Dragon, and had somewhat broken networking/error handling, so I rewrote most of it again and copied a bit of code over, into Wyvern.

Wyvern aims to:
  • Support big, multi-user setups.
  • Work sanely and reliably even in bizarre edge cases.
  • Be pretty fast.
  • Function conveniently for end users.
  • Allow complex automation.
  • Be relatively easy to connect other systems to.

Currently only a CLI client, automated IO interface, and backend for chest storage are available. This is enough for most setups, however.

Read more - and get installation instructions, etc - here.

It also has an optional acronym: "Working Yet Veritably Excellent Retrieval Network".

HydroNitrogen

Is it wireless? If so, I think it would be great if you'd make it wireless! (This concept is possible using a bound introspection module in a manipulator)

Currently I do not yet see the benefits of a CLI based Wyvern over GUI based SquidDev's Artist, which is very fast, does great searching and has a convenient GUI.

But perhaps wireless options may change that!
Please do collaborate on the ComputerCraft Wiki! Thanks!
Also like my post if you found it informative or helpful. This gets me to know what I should do more of. Thanks again!

osmarks

Aug 13, 2018, 06:30 pm #2 Last Edit: Aug 13, 2018, 06:34 pm by osmarks
Quote from: HydroNitrogen on Aug 13, 2018, 06:04 pmIs it wireless? If so, I think it would be great if you'd make it wireless! (This concept is possible using a bound introspection module in a manipulator)

Currently I do not yet see the benefits of a CLI based Wyvern over GUI based SquidDev's Artist, which is very fast, does great searching and has a convenient GUI.

But perhaps wireless options may change that!
It's designed for large setups, does not require an introspection module, has an API for automation (documentation coming Soon(tm)) and allows multiple terminals with synchronised data.

Wireless/introspection module access will come when I can be bothered to make the overlay glasses client, which will have the benefit of being able to display item icons. This will probably be around when SC updates to whichever version of plethora allows 3D overlay glass objects.

Also, searching is currently done via use of Levenshtein distance to check how close your search term is to the display name of items. I am looking into other approaches but haven't found any good ones, though this has its problems too ("wool" pulls not red wool but bows and coal due to... stuff).

Incin

Quote from: osmarks on Aug 13, 2018, 06:30 pmAlso, searching is currently done via use of Levenshtein distance
- snip -
though this has its problems too ("wool" pulls not red wool but bows and coal due to... stuff).

The problem is that it is easier to transform wool to coal than to red wool because to get to red wool you have to insert 4 characters, vs the 2 modifications to transform to coal.

To mitigate this, you should match against all (consecutive) substrings of length n of the strings you match against, where n is the length of your search string. If you need more information, checkout the search functionality used for the luadash documentation utility.
To understand recursion, you must first understand recursion.

SquidDev

Quote from: Incin on Aug 13, 2018, 06:52 pmThe problem is that it is easier to transform wool to coal than to red wool because to get to red wool you have to insert 4 characters, vs the 2 modifications to transform to coal.
Well, the bigger issue is that Levenshtein distance isn't a great metric for fuzzy matching. If I enter "cble", one'd expect to get "cobblestone". However, "chest" is actually closer!

I believe most fuzzy systems (and at least BSRocks' and Artist's) try to match up pattern letters against input letters in order and then score based on a series of factors like number of consecutive matching letters. You miss out on some benefits of Levenshtein (such as detecting transpositions), but it's not a great loss.
GitHub | CC:Tweaked: A ComputerCraft fork | Plethora: A peripheral mod

osmarks

Quote from: SquidDev on Aug 13, 2018, 07:04 pm
Quote from: Incin on Aug 13, 2018, 06:52 pmThe problem is that it is easier to transform wool to coal than to red wool because to get to red wool you have to insert 4 characters, vs the 2 modifications to transform to coal.
Well, the bigger issue is that Levenshtein distance isn't a great metric for fuzzy matching. If I enter "cble", one'd expect to get "cobblestone". However, "chest" is actually closer!

I believe most fuzzy systems (and at least BSRocks' and Artist's) try to match up pattern letters against input letters in order and then score based on a series of factors like number of consecutive matching letters. You miss out on some benefits of Levenshtein (such as detecting transpositions), but it's not a great loss.
If you have a link to some nice implementations of fuzzy search (mine is stolen from luadash) that I could use instead, please give me them.

SquidDev

Quote from: osmarks on Aug 13, 2018, 07:20 pmIf you have a link to some nice implementations of fuzzy search (mine is stolen from luadash) that I could use instead, please give me them.
This is Artist's implementation. It's incredibly simple (well, compared with the previous one) but seems to do a good enough job for most cases. There's some definite improvements you can make, such as treating _, camelCase and whitespace as equivalent, handling transpositions, etc... but it seems to work.
GitHub | CC:Tweaked: A ComputerCraft fork | Plethora: A peripheral mod

osmarks

Quote from: SquidDev on Aug 13, 2018, 07:34 pm
Quote from: osmarks on Aug 13, 2018, 07:20 pmIf you have a link to some nice implementations of fuzzy search (mine is stolen from luadash) that I could use instead, please give me them.
This is Artist's implementation. It's incredibly simple (well, compared with the previous one) but seems to do a good enough job for most cases. There's some definite improvements you can make, such as treating _, camelCase and whitespace as equivalent, handling transpositions, etc... but it seems to work.
What are the return values there?

osmarks

Aug 14, 2018, 09:35 pm #8 Last Edit: Aug 15, 2018, 04:27 pm by osmarks
Fancier fuzzy search has been implemented, though I may remove it if it doesn't work significantly better than the old one.

Coming Very Soon(tm), mostly because I need it right now: an automated IO (combination ME interface/import bus/export bus).

EDIT: The above is done!
I also implemented non-fuzzy search (prefix your query with !).