tag:blogger.com,1999:blog-8929346053949579231.post4454829999277514089..comments2024-03-23T00:59:24.057-04:00Comments on Sapping Attention: CorrelationsBenhttp://www.blogger.com/profile/04856020368342677253noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-8929346053949579231.post-54095831133511659952011-01-05T12:01:58.947-05:002011-01-05T12:01:58.947-05:00No case sensitivity anywhere in this system—I thin...No case sensitivity anywhere in this system—I think that's for the best.<br /><br />The next post should get at some of the questions question about discourses and what words to use. The reason I'm using TF-IDF scores here instead of simple frequencies is that it lets us combine different words into the same scale--so we could make a first stab at finding the correlation of "society"-"astors" to "society"-"evolution", say. It's probably a bit of a leap from word-usage-patterns to discourses, for sure.<br /><br />She and Government doesn't look that different than "evolution"-"society" at the raw level; I really need to figure what transformation to apply. Maybe I can just put up some log-scale graphs, which make this all much more obvious. (The problem is that you have to drop out all the books that don't have one of the words at all to do a log-transform, so you end up losing a lot of data. Maybe I shouldn't be worried about that--appropriately transforming between distributions is definitely one of my weakest areas.)Benhttps://www.blogger.com/profile/04856020368342677253noreply@blogger.comtag:blogger.com,1999:blog-8929346053949579231.post-39116408759406059942011-01-05T11:28:30.567-05:002011-01-05T11:28:30.567-05:00"She" and "Government" - I lov..."She" and "Government" - I love it! Could you show us what that most-negative of graphs looks like? I think that might help me figure out what to look for in those initial visual renderings (though I have an idea). <br /><br />There's a lot of reaching in here, but, as I've said from the beginning, that's the best. I'm with you on "Darwin" and "Evolution" (are your searches case sensitive, by the way?), but I wonder whether you might try "Evolution" and "Human," too? That is to say, are we sure hunting around for "society" is our best bet? It might be, but I'd like to see a few other candidates to compare correlation graphs. <br /><br />At a more general level, could you motivate the claim you make in your opening paragraph about this being at the intersection of ("between") discourses? You're not saying that the evolution discourses on diseases and the society discourses on the Astors become united, such that you get a discourse on Astor diseases...<br /><br />You're saying those discourses change, such that you'd need to account for (a) the shifts in each (from "Astors" to general society) and (b) how the new one relates to the old ones (starting to use a new sense of evolution to talk about this new sense of society). I know this isn't 100% lucid, but I'm eagerly awaiting your second correlations post and wonder if you might take a stab on this question (such as it is) therein...Hanknoreply@blogger.com