JavaScript is required to use Bungie.net

Destiny

Discuss all things Destiny.

This thread is inspired by another: view original post

Edited by freefall722: 2/17/2013 9:48:25 AM
25

Cleaning up the FFT data

[i]EDIT: We got an automation of this whole process up and running. Links below. Enjoy :)[/i] [i]EDIT: For Day 4 Bungie added a new wrinkle and split the output on different color channels. Following these steps now you'll have to threshold each color channel individually.[/i] [i]EDIT: For Day 6 they added rotation - so back to manual for now[/i] This is a description of how I made a clean image from the FFT output like the one attached (which used 8 separate images) Ok so the first thing is that you'll notice doing an FFT on an output image is that it gives you a pattern that looks very similar to the geomask image to the right - in fact it is exactly the same for every output/geomask pair. We can use this to know what data from the FFT is valid or not by using the geomask to mask the FFT output. Secondly we are going to assume that the final image will be black and white (no greyscale) so we can use a threshold to cut out the noise. Steps: *Download a output/geomask pair (I'm using these links - Mask: http://imgur.com/a/D1ORk , Image: http://imgur.com/a/0sno9 and the # in the link to make sure I download the right image from both for a pair) *FFT the output image (I'm using ImageJ for that then saving it out to a png for the remaining steps) *Apply the geomask as a mask to the FFT output (I used photoshop) --Open the geomask and copy all --Open the FTT output, unlock the background layer (double click it) --Got to Layer>Layer Mask>Reveal All --Go to the Channels window. Select and reveal the Layer mask channel --Paste the geomask image into the layer mask channel then hide the layer mask channel again --Go back to the Layers window and right click the image mask thumbnail and select Apple Mask (this makes pasting into other documents easier) *Use Threshold to remove the noise --Go back to the Layers window and make sure you have the layer thumbnail selected not the layer mask thumbnail. --Go to Image>Adjustments>Threshold - for the output given from ImageJ a threshold of 14 worked for every image (The important thing here is to have the threshold above the spike that represents valid data but below the small hump that represents the noise) From there I just kept pasting the resulting masked images into a blank psd with a light grey background. I used the light grey background so that I knew which sections I didn't have yet. This is very helpful since I could just go find another geomask that covered the sections I was missing. You don't have to do anything special to combine them since they are already cleaned and masked you can just keep adding them to each other. I'm sure this whole process could be automated as well - unfortunately for me the photoshop filter doesn't work on the mac version of photoshop otherwise this would be a pretty simply photoshop action to make. [i]EDIT: We got an automation up and running. New Links for all days: https://www.dropbox.com/s/goodiqip5g6x16l/finalMaskedOutputDay1.png https://www.dropbox.com/s/swcl4g03xyvos3l/finalMaskedOutputDay2.png Channels: [url=https://www.dropbox.com/s/tebx81swyh6hriv/finalMaskedOutputDay2_R.png]Red[/url] : [url=https://www.dropbox.com/s/kslecd24ftwx2fm/finalMaskedOutputDay2_G.png]Green[/url] : [url=https://www.dropbox.com/s/3zb1nki3j2vqv67/finalMaskedOutputDay2_B.png]Blue[/url] : [url=https://www.dropbox.com/s/wy6y49lmt2tj2v7/finalMaskedOutputDay2_BW.png]B&W[/url] https://www.dropbox.com/s/2p2gn23025o5vsg/finalMaskedOutputDay3.png https://www.dropbox.com/s/9lprjb1sl0j5ybb/finalMaskedOutputDay4.png Channels: [url=https://www.dropbox.com/s/ya54nju3py80lke/finalMaskedOutputDay4_R.png]Red[/url] : [url=https://www.dropbox.com/s/753z9d6iazkrh5v/finalMaskedOutputDay4_G.png]Green[/url] : [url=https://www.dropbox.com/s/8tuzb0426oa7s2w/finalMaskedOutputDay4_B.png]Blue[/url] : [url=https://www.dropbox.com/s/d5w605csj1ez86x/finalMaskedOutputDay4_BW.png]B&W[/url] https://www.dropbox.com/s/6u8114uvxph0noz/finalMaskedOutputDay5.png Channels: [url=https://www.dropbox.com/s/g299sie140ghw4o/finalMaskedOutputDay5_R.png]Red[/url] : [url=https://www.dropbox.com/s/x20m8m5psklpda8/finalMaskedOutputDay5_G.png]Green[/url] : [url=https://www.dropbox.com/s/zruwi7mwyinorev/finalMaskedOutputDay5_B.png]Blue[/url] https://www.dropbox.com/s/ouh14bh68ejev1e/finalMaskedOutputDay6.png Rotation (Split by hour): [url=https://www.dropbox.com/sh/a80l1mh2kpageyh/ReY4bepXwo]Folder[/url] https://www.dropbox.com/s/xn1n3vkhc38clfc/finalMaskedOutputDay7.png (I'll upload more after monitoring for a while) Enjoy :)[/i]
English
#Destiny #arg

Posting in language:

 

Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Edited by freefall722: 2/19/2013 3:18:14 AM
    Now that I've had some time to take a break I've gotten the automation code up. I was going to set up a git but since this Dropbox thing has been working out so well last week I just put it up there. (Feel silly because I could have done that awhile ago.) https://www.dropbox.com/sh/2tmhcdwhezb0gma/nnMNMT1ef1 I should also note (to spare my programming pride) that this code wouldn't win any design awards. It was written as quickly as I could type it - so it doesn't have much error checking, memory management (it definitely leaks), comments, or organization. It does do some cool things I'm proud of though, but because of the above I wouldn't use it as way to learn Obj-C ;)

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Edited by freefall722: 2/17/2013 9:47:45 AM
    New Links for all days: (Day 7 Craziness!) https://www.dropbox.com/s/goodiqip5g6x16l/finalMaskedOutputDay1.png https://www.dropbox.com/s/swcl4g03xyvos3l/finalMaskedOutputDay2.png Channels: [url=https://www.dropbox.com/s/tebx81swyh6hriv/finalMaskedOutputDay2_R.png]Red[/url] : [url=https://www.dropbox.com/s/kslecd24ftwx2fm/finalMaskedOutputDay2_G.png]Green[/url] : [url=https://www.dropbox.com/s/3zb1nki3j2vqv67/finalMaskedOutputDay2_B.png]Blue[/url] : [url=https://www.dropbox.com/s/wy6y49lmt2tj2v7/finalMaskedOutputDay2_BW.png]B&W[/url] https://www.dropbox.com/s/2p2gn23025o5vsg/finalMaskedOutputDay3.png https://www.dropbox.com/s/9lprjb1sl0j5ybb/finalMaskedOutputDay4.png Channels: [url=https://www.dropbox.com/s/ya54nju3py80lke/finalMaskedOutputDay4_R.png]Red[/url] : [url=https://www.dropbox.com/s/753z9d6iazkrh5v/finalMaskedOutputDay4_G.png]Green[/url] : [url=https://www.dropbox.com/s/8tuzb0426oa7s2w/finalMaskedOutputDay4_B.png]Blue[/url] : [url=https://www.dropbox.com/s/d5w605csj1ez86x/finalMaskedOutputDay4_BW.png]B&W[/url] https://www.dropbox.com/s/6u8114uvxph0noz/finalMaskedOutputDay5.png Channels: [url=https://www.dropbox.com/s/g299sie140ghw4o/finalMaskedOutputDay5_R.png]Red[/url] : [url=https://www.dropbox.com/s/x20m8m5psklpda8/finalMaskedOutputDay5_G.png]Green[/url] : [url=https://www.dropbox.com/s/zruwi7mwyinorev/finalMaskedOutputDay5_B.png]Blue[/url] https://www.dropbox.com/s/ouh14bh68ejev1e/finalMaskedOutputDay6.png Rotation (Split by hour): [url=https://www.dropbox.com/sh/a80l1mh2kpageyh/ReY4bepXwo]Folder[/url] https://www.dropbox.com/s/xn1n3vkhc38clfc/finalMaskedOutputDay7.png (I'll upload more after monitoring for a while)

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Edited by freefall722: 2/16/2013 8:06:54 AM
    Day 6 is Up! (these links will always point to the most recent version): Full: https://www.dropbox.com/s/ouh14bh68ejev1e/finalMaskedOutputDay6.png B&W: https://www.dropbox.com/s/yqjty0gba94jrq9/finalMaskedOutputDay6_BW.png R: https://www.dropbox.com/s/05x1xtn527dltxx/finalMaskedOutputDay6_R.png G: https://www.dropbox.com/s/sy5vaig94t6u2f9/finalMaskedOutputDay6_G.png B: https://www.dropbox.com/s/jlsusd6w2tm216a/finalMaskedOutputDay6_B.png C: https://www.dropbox.com/s/21pakcql01t64es/finalMaskedOutputDay6_C.png M: https://www.dropbox.com/s/ap5ecj15lmsygbw/finalMaskedOutputDay6_M.png Y: https://www.dropbox.com/s/6nojkqnidg92yug/finalMaskedOutputDay6_Y.png

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Tried doing this my self, but I couldn't the Mask to work. I got the fourier transformations to work on the output images but I'm not sure I get the geomask section. Probably because I'm using elements.....-.-

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

    3 Replies
    • Edited by freefall722: 2/14/2013 4:50:20 PM
      Bungie added a little wrinkle to Day 4 by storing information in separate color channels like the example above. I'm rewriting the automation to handle this. Thanks to SepheusIX for his help with this. Edit: Automation is back up! https://www.dropbox.com/s/9lprjb1sl0j5ybb/finalMaskedOutputDay4.png Edit: B&W version: https://www.dropbox.com/s/d5w605csj1ez86x/finalMaskedOutputDay4_BW.png

      Posting in language:

       

      Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      10 Replies
      • Dropbox looks blank? Is this intentional?

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

        2 Replies
        • [url=http://www.dropbox.com/s/hdvovhb6wy4zpdm/Day_3.zip]Here[/url] is a collection of 25 outputs and geomasks, sorted by time taken. Hopefully someone finds it useful. Not sure if everyone has got their scripts on yet. After attempting to ananlyse them, at threshold=14, everything is white, but I doubt much has been revealed yet.

          Posting in language:

           

          Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

          1 Reply
          • Link is up now: https://www.dropbox.com/s/9lprjb1sl0j5ybb/finalMaskedOutputDay4.png I was weak and feel asleep - missed the first few hours of data. If anyone has those output and geomask pairs I can add them in.

            Posting in language:

             

            Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

            1 Reply
            • The title seems to be a number sequence followed by .jpg mirrored upside down as well

              Posting in language:

               

              Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

            • I am just getting white after the threshold. Am I doing something wrong or is the image just not loaded enough yet, considering it's only been up for half an hour?

              Posting in language:

               

              Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

            • The image on the left with the writing gets clearer depending on the amount of people on the site right?. So is it possible to get the writing clear enough to read and not needing this process to find the jpg page? Just wondering.

              Posting in language:

               

              Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

            • Edited by Skittles x: 2/13/2013 6:35:35 PM
              Threshold of 14 doesnt seem to work for me... or am I doing it wrong? edit: [url=http://i4.photobucket.com/albums/y128/nick53182/Thresh-01.png]This is what my output looks like from the Day 1 images... Fourteen doesn't work here either. I need a value closer to 100. (if I'm looking at it correctly...)[/url]

              Posting in language:

               

              Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

              2 Replies
              • Edited by Kvaener: 2/14/2013 1:43:00 AM
                This is going to make my life Hell in the morning...

                Posting in language:

                 

                Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                1 Reply
                • Might want to watch out - the left image isn't fully loading right now. Might throw off some automation process. Just thought I would warn you. (Seems as though multiple people are having this issue)

                  Posting in language:

                   

                  Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                  4 Replies
                  • Any chance of IP spoofing the web calls to make it appear like various peoples from across the globe are active?

                    Posting in language:

                     

                    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                    1 Reply
                    • When I paste the FFT image into the layer mask the original layer turns in to grey and white squares leaving only the mask any way to fix this?

                      Posting in language:

                       

                      Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                      1 Reply
                      • http://imgur.com/iLGgE1a http://imgur.com/P3rTXVU Do not know how to upload to your folder so anyway here it is

                        Posting in language:

                         

                        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                      • https://www.dropbox.com/s/9a9a1eec15fgwrf/finalMaskedOutput.png Me and Recon Ocelot spent a few hours automating this process. Enjoy :) (The above link will always link to the latest version)

                        Posting in language:

                         

                        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                        10 Replies
                        • I'm having some incredible issue dealing with installing the photoshop plugins. Where and how can I install the FFT filter plugins for my Photoshop CS5 x64bits.

                          Posting in language:

                           

                          Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                          1 Reply
                          • Edited by Ockeghem: 2/13/2013 2:25:18 PM

                            Started a new topic: Destiny ARG day 2 - solution / QR code link?(1 Reply)

                          • Edited by Beorn: 2/13/2013 4:05:04 AM
                            Great write-up! Thank you, freefall722. This is a more elegant of handling layer combining than I was attempting this morning. :)

                            Posting in language:

                             

                            Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                          • I keep getting black and grey squares!

                            Posting in language:

                             

                            Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                          • Edited by ncsuDuncan: 2/13/2013 1:19:48 AM
                            Excellent writeup! I'll add a link to this on ARG page so everyone else can give it a try tomorrow! (Assuming the FFT method is still valid, of course.) ;)

                            Posting in language:

                             

                            Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                          • Lovely, thanks!

                            Posting in language:

                             

                            Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                            9 Replies
                            • Much appreciated. With multiple people running this game at once, we can just overlap our pictures until we get a clear image.

                              Posting in language:

                               

                              Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

                            You are not allowed to view this content.
                            ;
                            preload icon
                            preload icon
                            preload icon