Golden Age of Books

Golden Age of Books (Atlantic article)

Interesting, the same could be said for music composition up to a certain point in history.  Prior to the 20th century, mass market music entertainment more likely were in the form of folk music arts, if you were lucky to find a fiddle player, pianist (if something like the piano even existed and someone in the village could even afford this instrument), or anyone else, and orchestras, ensembles, and chambers were more so for wealthy clientele.  As to mechanical devices, maybe the music box, or some other music contraption having existed if you could afford might play some medley of one sort or another.

Industrialization no doubt would have an impact in terms of publication, but who could imagine that writings could be more prolific then what should be found today?  Maybe one has it in mind that publishing circles were more selective then today, relative to automated posting where information from such could be expected more likely filtered and screened, neither in the sense of human comprehension or worthiness for markets in terms of selling points, but amongst search relevance.   If you were looking outside of formal marketplace publication sites for data, your search engine were probably the best library indexing catalog search that should exist.  I would offer here, generally in recent years I have had more successes finding what I were looking for as opposed to not finding in terms of publication.  As to marketplace publication, its not that anytime past wouldn't presented the same opportunities in terms of qualities of writing, if people were given the same opportunities as found today.  Selectively speaking, there might be a handful of  'arena' style rock authors attracting so attention and popularity out of a given higher populace pool of writers, and then part of this could be in relation to the target audience.  I've found myself having just opened Umberto Eco's Foucault's Pendulum recently which contains a densely packed narrative infused with so much history.  Here it is interesting at least if you know something about the author himself having as much higher educational training, and alongside this having the interest in history at that which should lend itself so well in writing.  Good writings are not easy to come by always.  Some could be simple in terms of stylistic presentation, while others are much more densely elaborate.  Even the more common mass market author's may have researcher(s) having provided aid material to supplement the structure of fictional narratives, and what this means relative to the typical online posting such as mine or anyone else relying more so on the resource of self knowledge is that gaps between information, knowledge, and expectations at times could be self evident.  If you happen to be a professor having so much the wisdom of previous experience in life regarding your knowledge base, may certainly have positive effect with respect to the establishment of a base to artistically work from, but then exceptionally sometimes I have found excellent voices that were simpler in terms of narrative style that I have liked just as well...like Saramago as one example.  Maybe we are inclined to say books have declined given so much publication to all else in the marketplace that should trivial in mind and neither possessing the attributes of timeless qualities which should lend to translation throughout the ages, but really in the future, its not hard to imagine, works appearing with some level of importance, relevance and context to any future generation that were uniquely conceived relative to any time past.  In any event, I imagine finding such works not being so much difficult either, given so much technology to aid in the process of civilizations selecting such work.  We just have a perception given of decline, given so much at the buffet.
 

Increasing site traffic

     Hmm, mine tends to a lot flatter at the moment, but I were generating something like several hundred hits a day at some point previously in the past and probably close to the one thousand hits per day threshold, before I did a bit of self sabotage to my site, and caused traffic fallout removing a lot of previous posts.  At presents I am slowly increasing trending at the site as posts are actively increasing. 

The things that I could tell increasing site traffic:

1.  While generic things can sell in terms of hits.  If you are competing with the opinions of others and aren't so distinct in your messaging, you'd probably need some compensating hooks, like an existing social network built to aid in traffic generation.

2.   Whether people truly like the opinions at your site or not...granted you'd see at my site absolutely no followers, but you still may generate traffic, even if people aren't so interested in commenting.  Commenting in any event, I would think outside of certain social networks may be far and few between.  Case in point:  certain Youtube videos could have 30,000 + hits easily in a few months or years, and maybe just a handful of comments posted based upon the level of moderation provided.

3.  Finding something original to post can be a matter of reaching an audience where there are gaps in data and information on a particular 'search engine' query.  If you happen to see few decent comments or relevant posts on a topic that you feel you might have some expertise in sharing information regarding, you may be more likely to get higher ranks if  your data and information has strong relevance alongside being useful in the context of a given search query.  There's nothing crazy or 'wacko' about this logic, correct?

4.  While it may help already having social networks established in getting a new site up and running in terms of promotion.  It may not be everything.  I generate site traffic worldwide.  Albeit predominantly from the North America, followed by Russia, followed by India, followed by countries in Europe, followed by South American countries.  I don't formally have direct links or established communications with any that read my posts, and generally I'd rather keep, instead of soliciting obviously (albeit except with respect to mentions in this post) those interested in information, site traffic usage private generally speaking.

5.  I have found more strongly outside of social media postings which are generally ranked poorer in my opinion in so far as expressing anything of resource use, that my data/resource related posts tend to rank the highest in site hits.  While occasionally some odd comment about movies, books, and what not generated some interesting site traffic hits, the highest ranking hits are those topically related to: programming information, especially where again gaps of information exist in so far relevant data furnished through search engine queries, but there could be differences in the type of resource information provided and where traffic tends to go with respect to the type of information being sought.  It may be harder for me to talk, for instance, at my blog site about specifics of a biological organism where others might be generally attracted to destination sites like Wikipedia, but in terms of example data/resource information that goes beyond the scope of what Wikipedia might host in so far as relevant data and information, blog sites could fill some destination gap continuities?

  Its hard to gauge site traffic relative to content at sites that are more casual in terms of webmaster tools.  This would include google+, and facebook from what I could tell.  Albeit, supposedly friends network lists should be an implicit indicator, this isn't always true in so far as generating site interests.  While promotional links maybe of aid through social network sites, not sure how well this ranks in generating new traffic relative to existing traffic channels.  This would all depend I imagine on the relation of relevance to social networking site search queries.  Otherwise, unless you've made a big list of friends through a given social network site you might have stagnant site traffic growth for added traffic on redirects?

 In terms of fluidity and dynamics of site structures.  Artists might like the blog for any number of reasons, for example, site customization, and I've seen occasionally links to MySpace (despite the general negativity surrounding this once Facebook like site) especially where artists might like some form of content and presentation controls that hadn't existed at other social networking sites, but some are will perhaps to settle with the idea that as long as the 'site' hadn't provided too much visual obstruction to the presentation of any particular media, its also no surprise seeing any number of artists that do use social networking sites.   This is to say its not hard to imagine that people go where trending and opportunity are furnished, and many social networking (despite its own sense and structure of representation) tend to have their own unique style and presentations with respect to media content.  One of the more visually interesting sites again were So.cl if you were sharing with others a set of personal interests, especially easy with respect to posting in some visual sense any number of diverse sets of media in collage, granted these are of fixed form type from what I could say based upon presented format through the So.cl site. So.cl is convenient for providing a unique style here relative to other sites, but it seems outside of personal queries and posts, its yet another more refined controlled media form relative to the blog.

And then if you were wanting more literary, word intense presentations, alongside greater custom graphical media presentations outside social networking sites, the blog is probably a better choice.  Most other social networking sites tend to be less word rich, and much simpler in terms of personal expressions.

Personally I've made use of my blog in conjunction to personal interests, activities, thinking, and writing in the extended sense going beyond the other social media sites.  While I have posted sometimes at length at these other sites.  Unfortunately one may run (aside from the same being said at blog sites) saying too much may not necessarily generate new interests, unless you have already established interests that like your voice.  If you can create hooks in your writings, outside of being strictly utilitarian, in your site use, you might be more successful then others.

Worst posting here in site:  Layman's glance at gravitation.  A very small population of hits here.
 I suspect poor ranking probably in competition to similarity of topic in discussion or relevant information.  Layman talk and popularizing more complex science subject matter may not provide you an audience unless you have an established name.  If you are saying the same things that a Carl Sagan or any other popular scientist is saying, chances are you'd be lost in relevant discussion unless topically you were discussing topics that weren't as common relative to the other crowd that would be competing in so far as queried search results.  Generalities in discussion for already popular subject matter make for stiffer competition in terms of successful ranking, I'd guess.

You may not need be a destination site for readers to generate traffic, or at least you could be somewhat ala carte oriented in presentation where reader's generally find something useful or interesting to search by when crossing paths with your site.  The disadvantage to ala carte presentation is that your interests and thinking, especially outside of general interests of topics, may not always attract enough the attention of a reader.  Thus if you were by example discussing Quantum mechanics which one reader had thorough knowledge of, and then in the next post managed to be discussing in depth, beyond lay terms, an engineering related topic you might not generally hold destination traffic other then occasional sense.  Either way you might see traffic but mostly these could be on the basis of queried searches that were specifically topical to a particular post.

My blogs tend to be a think pad for me, and added public storage...alongside cloud stuff here, so personally I am not necessarily concerned with attracting readership necessarily nor depend upon it in terms of monetization.  Thus if need to answer some questions related to general interests that I might already have which Google hadn't readily solved for me so conveniently, I could at least scan my own personal database of storage for answers to these questions.

Finally as to the other crowd half that were at times fairly juvenile in any number of responses, like not really having too much to say other then something that should seem somewhat clever but not really, or persistently responding without so much opinion otherwise other then to be clearly personally insulting and having used any sort of personal language at that, mostly those sites I'd imagine are a: why bother?  Usually sites like this that are well established like this sure enough may have their venues and enough social interchange keeping them afloat for any length of time but I'd laugh if originality were suggested.  Sure trivial destinations could seem okay for awhile, but really its not my cup of tea.  I'd suggest anyways that blog and market sites are probably over saturated with any number of sites or personal forum poster's that litter any number of posts that would generally be quite boring if it weren't something of the language, however obnoxious that were supposedly to illicit a response.  I read an editorial that best described this:  If you are writing about your bodies physiological responses and making this availed in the public sense, its not really original, everyone defecates, everyone gets sick from time to time, everyone gets sad or angry, and what not.  Its another matter if you explain yourself and you differentiate yourself relative to others as to motivations here, and not merely for the sake of attracting reaction.  This is to say, while there are some sites that are generally successfully popular selling the trivial and superficial interests, most probably fall outside the successful category.  If it weren't for some other hook points that would keep an audience interested in a particular site.  Trying to sell the same stuff that others are selling in competition isn't great in my opinion, especially when there isn't much to be added in terms of diversity of opinion that were interesting.  I know you probably see people over at Ms complaining that whatever site interests generated here were inordinately unfair as to search engine biases (where maybe some of this at times were the relation between intellectual property rights and open source industries), and 'fallout boy' isn't selling as well as they used to...granted there's Rhianna...

   Laughing, yeah, taking interest in technology means you need to be tech's demographic?

Yeah apparently at other social networking site's there seems to be so much advice coming with respect to the idea of one's motivation.  Fairly amusing for that side of culture that reads so much more in glossary ways.

Over the years I've tired of serving the blog in the same ways which means taking up some other constructive efforts here.  Generally I avoid so much political comment here while you might on occasion see a flippant comment here or two.  Unless you have something worthwhile to say that weren't in league to any number of the same tit for tat lame insults that have been used so tirelessly in the past, political blogs generally are lame.  Again as in above the same rule as always applies, maybe some ex cia agent runs a site that happens to have any number of his own sets of opinions, but at least you hope the site is informative as opposed to just another cheerleader saying the same things.  I've seen decent local political blogs that neither attempt so much taking up any niche of blog posts that cover topically things like National politics that so much of media attention seems governed at with respect to politics in general, and if you are decent and have the no how, more power to you, you may have a good starting point for a local niche and possibly attracting some readership outside your local area of coverage.  As to national blogs a personal endeavor with lackluster social connections is probably not great.   Writing at length about the opinions and reporting of others all the time, if you don't have original source material to present on your own, is just added time in someone's reading schedule.  I'd make a prediction that there have been for sometime plenty of editorials predominating likely at times over the readership of regular news and this could be trending at some point in the future downward.  It used to be that late night comedians got in on the act (Comedy central) of taking up this lampoon of a media market niche...which ended up in one funny comment from popular comedian commentator: you know its sad when a regular media outlet, constantly talks about you talking about them, and then we are talking about a site generally attracting more traffic then myself, and having really at least in so far as the selection of materials  little to say at times on politics otherwise.  In other words, while being selective and maybe self absorbed in your writings is okay, if you aren't original, its probably going to be sad news for your blog site if you don't have lots of money to back up your social media outlet, and in this given age of much being availed in so far as information online, while it seems neat re posting stuff and having little to say...I know I've been guilty as charged....chances are everyone probably has something neat to post.  Preach to choir, political funnies...hmm, sort of lame really in my opinion, especially being spoon fed this through out the day and given little else in reading.

Then there's the crafts/pop art crowd, its okay stuff out there, not a place that I would find myself dwelling for any length of time, no offense.  Blogs of these sort need not be popular and generally I couldn't criticize for being any better with respect to artistic abilities.  Generally speaking some post rather lamely something to the effect: you have no business posting on line, get lost and what not, and don't accept mediocrity.  Only more self evident here that Snooki has a role to play in society's culture.  Sure one can rail counter culturally against so much of this society, but really just one voice amongst many.  I'd have to compliment craft sites which seem ever bit as self absorbed in their scrap booking work whose relevance were probably extended more likely to close family members and friends more likely.  If you are perfectly happy and content with your blog site, there's probably no need to read further into this post about generating site traffic.  You could probably laugh at those running a post on mediocrity and art, accept no substitutes, and buy ever bit into their market form and culture likewise.  If its interior design and Electrolux, its not worth the thousand dollars spent for the shoddy engineering of a glossy looking machine that were probably for such households that spend little time actually using such machine.  If you think of higher dollar popular arts, or things which seem so much less mediocre in mind: isn't this so much perception?  Aren't we sold on a bill of goods at times?  Does paying for me mean that something is really better in the aesthetic and utilitarian sense?  And then often times, one wonders whether there isn't something of lobby that exists at these sites reflecting inherent self promoting culture.


There's the family blog which is outside of close friends and family members will probably be limited for most.  I've visited some with personal biographies, interests, and happenings.  Honestly though, if family appears like my own in many respect, I'd relate but generally most sites that I have run across are more limited with respect to their activity.  I read excellent publication surrounding personal circumstances having happened and the subsequent aftermath of events therein.  Seemed that the blog ended well, sparing the reader of any number of trivial events that should merely serving in other circumstances as writing for its own sake.  Maybe it is that we are so conditioned by way of fiction to the notion that the interesting biographies have some story to tell, something of parable, or anything regarding a dynamic exchange between one's environment and personal circumstances.  Some are simply informative with respect to the generalities of a given family.  Having mentioned in context any number of personal happenings and events but not necessarily designed or oriented in the sense of facilitating a story.  These need not always be the case, but the audiences reading may in some cases be limited.  Graduations, marriages, and births that say little more will probably limit the scope of any number of reader's attention.  Lots of family's have births, marriages, and graduations.  Having any number of personal experiences that express some diversity relative to these more common experiences may likely attract greater readership attention.  At least I personally know that I find myself more likely to read writings relating to personal biographies which provide the reader a sense of the character that is being written about.  One need not provide too much information, or you could try your hand at fiction...however poor or not so poor you were.  You might be successful and maybe not, but then maybe its an excellent side project to try writing some personal biography, or fiction otherwise.  I've recently tried short story fiction because it seems a lot easier, and even if I am not so good, it provides opportunity to change prospectively writing styles.  Anyways, even if you aren't so good, who cares the days right?  Its not like you have to be an expert, or good to take a creative writing class or attempt writing and the more attempts could mean better success, or it could mean something like my past attempts at the viola, sort of off key and really not so good, but does it matter?  Probably not.


As to site traffic after a day.  Any number of hits drawn the first day, tend to be common like other posts, indicating something of a destination audience with traffic tapering off in time.  There have been exceptions to this rule with some of my posts which have generated over longer spans of time site traffic...usually again I would suspect because of existing in a niche amongst other much poorer ranked relevantly furnished data.  Microsoft can complain all they want, but if they think that for instance site traffic weren't generated because of correlative relevance here, they'd be sadly mistaken.

Chances are you may not likely have a rock star oriented site.  If you looking for millions of hits a day, likely it helps being established.  There may be an appreciable site audience common entertainment: humor, popular music, and then movie media.  One could offer aside from much formulation and structure provided, that unless your site weren't entertainment oriented, resources sites unless rich in diversity are probably going to be more selective in terms of appreciable audiences.  I've sort of made my site into a sort of personal hobbies and interests sort of place.  This isn't to say that there isn't an audience that connects out there with some of my interests, but as individual alone I may be limited in terms of those connecting with the things that I am doing, and if this is what you are aiming for, so much the better success to you.  At least you hadn't generally need to cater so much in the subtractive sense to personal interests for the sake of general entertainment, or  in other words just because your site isn't arena rock, doesn't mean that its not worthy.  Maybe you like being eccentric, obscure, arcane, less commonly thought of...or could care less one way or the other, and then some potential audience may find you in time...at least for some artist this weren't found in their own lifetimes.  Thus far I've generated over thirty thousand hits, if it gives an idea between the relation of subject matter and site traffic.  There are people that may find the things you say relevant and interesting.  The world is a diverse place.  The simplest rule that I have found.  If you can't find an answer to a question with your search engine, and you know the answer, post something about it.  You may find yourself getting ranked better then you might suspect, and you've not only helped yourself but others in posting.

Hash libraries and encryption

      Albeit not a subject matter that I have significant expertise, and generally while I would think that hashing simple password were still not a great idea.  I'd demonstrate briefly why:

Okay so if you have python installed, you'd probably have access to some basic encryption stuff here. 

You could do some basic password encryption as follows:

import hashlib
m = hashlib.sha1()
m.update(b"abcde")
print(m.hexdigest())

you'd get some output like

>>>
03de6c570bfe24bfc328ccd7ca46b76eadaf4334

which is pretty large looking bit of string that could take awhile to cipher.

The problem is that if I using a set of common password strings, hashed the same string 'abcde'

I'd get the same hash ed return value, and I'd have solved the cipher ed code quickly, but the good news is that there are a number of hashing routines (different encryption methods) other then .sha1() for instance,  and a hack would have to have any number of common passwords translated by way of any number of hash methods , so you might more safe when it comes to the gamble of hashing a very easy password, and given likely inter web securities, hacks might have so many chances before being locked out potentially of password attempts, but it still might be wise to pick a more difficult password that included mixed case alpha numerics that were greater then several character/digits in length.  Now if you had some custom hashing encryption method that were decent enough, I'd imagine you might be safer with even a easier password, but at the moment.  If you are using more common encryption software that hadn't generated unique keys for the hashing method that only you had access to...the above method, for instance, wouldn't be so great for simple passwords in my opinion.

The following is a variation from a method that I posted online.  While this creates a random sequence added to the 5 places of the encrypted password, it customizes and mixes the hashing process for password generation.  Here you presumably need to hold onto the saltkey data key.  You'd need to likely store this somewhere on your personal computer or write it down.  Its only 5 places/digits in length here, and then make sure it is recalled back in the class function either from memory or adding to a global variable in your python file...real world implementation would be slightly different also having disabled the set_password function in the encryption class upon instantiation.  Generally speaking I would tend to think using the method, your very simple password saved alongside your personal saltkey would be a much harder encryption to break.

import hashlib
alphanumer = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
              'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',
              'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',
              'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V',
              'W', 'X', 'Y', 'Z', '1', '2', '3', '4', '5', '6', '7', '8',
              '9', '0', ' ', '.', '!', ',', '@', '#', '$', '%', '^', '&',
              '*', '(', ')', '-', '_', '+', '=', '[', ']', '{', '}', ':',
              ';', "'", '"', '?', '/', '>', '<']
class encryption:
    def generate_privatekey(self):
        import random
        
        randomkey = str(random.random()) + '$' + str(random.random())
        randomkeyutf16 = bytes(randomkey, 'utf-16')
        #store this salt, alongside your raw_password
        salt = hashlib.sha1()    
        salt.update(randomkeyutf16)
        saltkey = salt.hexdigest()[:7]
        return saltkey
        
    def encrypt_message(self, raw_message):

        
        privatekey = self.generate_privatekey()
        encryptmessage = ''
        for letter in raw_message:
            preencryptpart = privatekey + letter
            pencryptutf16 = bytes(preencryptpart, 'utf-16')
            hshlb = hashlib.sha1()
            hshlb.update(pencryptutf16)
            hsh = hshlb.hexdigest()
            encryptmessage += hsh
        return encryptmessage, privatekey

    def gen_decrypttables(self, privatekey, encryptmessage):
        #this works in a limited context for western latin based scripts
        #you'd have to make sure to import the correct character set for
        #alternate languages.  I'd recommend encrypting into the private
        #key a language code to reduce computation times on character sets.
        decryptiondict = {}
        for letter in alphanumer:
            keystring = privatekey + letter
            bkeystring = bytes(keystring, 'utf-16')
            hshlb = hashlib.sha1()
            hshlb.update(bkeystring)
            hsh = hshlb.hexdigest()
            decryptiondict[hsh] = letter
        decryptmessage = ''
        count = 0
        while len(encryptmessage) > 0:
##            print(decryptmessage)
            for hsh in decryptiondict:
                if encryptmessage.find(hsh) == 0:
                    decryptmessage += decryptiondict[hsh]
                    hshlen = len(hsh)
                    encryptmessage = encryptmessage[hshlen:
                                                    len(encryptmessage)]
                    break
            count += 1
            if count > 9999999999999:
                break
            
        return decryptmessage
        
    def set_password(self, raw_password, saltkey = None):
        import random
        randomkey = str(random.random()) + '$' + str(random.random())
        randomkeyutf16 = bytes(randomkey, 'utf-16')
        #store this salt, alongside your raw_password
        if saltkey == None:
            salt = hashlib.sha1()    
            salt.update(randomkeyutf16)
            saltkey = salt.hexdigest()[:5]
        preencryptpass = saltkey + '$' + raw_password
        pencryptutf16 = bytes(preencryptpass, 'utf-16')
        hshlb = hashlib.sha1()
        hshlb.update(pencryptutf16)
        hsh = hshlb.hexdigest()
        self.password = '%s$%s' % (saltkey, hsh)

    def check_password(self, raw_password):
        """
        Returns a boolean of whether the raw_password was correct. Handles
        encryption formats behind the scenes.
        """
        
        saltkey, hsh = self.password.split('$')
        preencryptpass = saltkey + '$' + raw_password
        pencryptutf16 = bytes(preencryptpass, 'utf-16')
        hshlb = hashlib.sha1()
        hshlb.update(pencryptutf16)
        hsh2 = hshlb.hexdigest()
        return hsh == hsh2

    def __init__(self, rawpassword):
        self.set_password(rawpassword)
        
rawpass = 'abcdeabcde'
a = encryption(rawpass)
print(a.password)
b = a.check_password('abcdeabcde')
encryptmessage, privatekey = a.encrypt_message('The cat walked home.  The eagle has landed.  The eagle has landed.')
print('encryptmessage: ', encryptmessage)
print('privatekey: ', privatekey)
decryptmessage = a.gen_decrypttables(privatekey, encryptmessage)
print (decryptmessage)

Basically a bit untested but example of thought code using an encryption method, couldn't speak as to its relative strengths or weakness here, but at least it provides a working encryption example here. Variations that I could think on the method above: extend the private key (creating more higher permutation sets on the encrypted message, and then create a modulus set that refer to the select truncation of the encrypted script object in an ordered way. For the later example, consider this you could instead choose a fragment of the original encrypted message, and then construct yet another set of random keys to be added to the present private key which instructs the user by way of ordering to search the overall hash object say with a 4 position truncation of the original message say starting at index position 3 of the original message. Thus a modulus set added to private key could look like: 02345. Which is  electing index 0 for the emessage fragment, index position 2 for the second,..., index 5 for fifth position. Then repeating this cycle of indexing on the whole of the message for letter encryption. Generally speaking the method above with a 7 position length private key, assuming a hack knew exactly the size of the private key to be generated, via brute force method could take 91^7 = 51,676,101,935,731 computations which is already up there in so far as code cracking algorithms. Adding larger private key sets extends this so that traditional brute force methods I would imagine are wholly impractical, and adding the modulus key for fragmenting parts of the original encryption method, reduces the size of the message more practically to that of the original.

Thus a stronger encryption version could look like this:

import hashlib
import random
alphanumer = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
              'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',
              'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',
              'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V',
              'W', 'X', 'Y', 'Z', '1', '2', '3', '4', '5', '6', '7', '8',
              '9', '0', ' ', '.', '!', ',', '@', '#', '$', '%', '^', '&',
              '*', '(', ')', '-', '_', '+', '=', '[', ']', '{', '}', ':',
              ';', "'", '"', '?', '/', '>', '<']

class encryption:
    def generate_moduluskeyset(self, epart, moduluskeyset):
        #where epart is the encrypted partition
        epartlen = len(epart)
        mk = random.randint(0,len(epart)-6)
        moduluskeyset +=  str(mk) + ' ' 
        return moduluskeyset

    def generatemsetlen(self, epart):
        #7 set length, although you could extend this if you wanted
        #to.
        setrange = range(0, 7)
        moduluskeyset = ''
        for i in setrange:
            moduluskeyset = self.generate_moduluskeyset(epart,
                                                        moduluskeyset)
        return moduluskeyset, moduluskeyset.split(' ')
        
    def generate_privatekey(self):
        
        randomkey = str(random.random()) + '$' + str(random.random())
        randomkeyutf16 = bytes(randomkey, 'utf-16')
        #store this salt, alongside your raw_password
        salt = hashlib.sha1()    
        salt.update(randomkeyutf16)
        saltkey = salt.hexdigest()
        return saltkey

        
    def encrypt_message(self, raw_message):

        
        privatekey = self.generate_privatekey()
        encryptmessage = ''
        capture = False
        lindex = 0
        for letter in raw_message:
            preencryptpart = privatekey + letter
            pencryptutf16 = bytes(preencryptpart, 'utf-16')
            hshlb = hashlib.sha1()
            hshlb.update(pencryptutf16)
            hsh = hshlb.hexdigest()
            if not capture:
                moduluskeyset, mset = self.generatemsetlen(hsh)
                mset = mset[0:len(mset)-1]
##                print('e mset: ', mset)
                capture = True
                modk = len(mset)
            mpos = lindex % modk
##            print('mset ', mset[mpos])
            mstart = int(mset[mpos])
            mend = int(mset[mpos]) + 6
            encryptmessage += hsh[mstart:mend]
            lindex += 1
        privatekey += ' ' + moduluskeyset
        return encryptmessage, privatekey

    def gen_decrypttables(self, privatekeyg, encryptmessage):
        #this works in a limited context for western latin based scripts
        #you'd have to make sure to import the correct character set for
        #alternate languages.  I'd recommend encrypting into the private
        #key a language code to reduce computation times on character sets.
        keytables = privatekeyg.split(' ')
        privatekey = keytables[0]
##        print('private key: ', privatekey)
        mset = keytables[1:len(keytables)-1]
##        print(mset)
        modk = len(mset)
        decryptiondict = {}
        for letter in alphanumer:
            keystring = privatekey + letter
            bkeystring = bytes(keystring, 'utf-16')
            hshlb = hashlib.sha1()
            hshlb.update(bkeystring)
            hsh = hshlb.hexdigest()
            decryptiondict[hsh] = letter
        decryptmessage = ''
        count = 0
        
        while len(encryptmessage) > 0:
##            print(decryptmessage)
            mpos = count % modk
            mstart = int(mset[mpos])
            mend = int(mset[mpos]) + 6
            for hsh in decryptiondict:
                hshtrunc = hsh[mstart: mend]
                if encryptmessage.find(hshtrunc) == 0:
                    decryptmessage += decryptiondict[hsh]
                    hshlen = len(hshtrunc)
                    encryptmessage = encryptmessage[hshlen:
                                                    len(encryptmessage)]
                    break
            count += 1
            if count > 9999999999999:
                break
            
        return decryptmessage
        
    def set_password(self, raw_password, saltkey = None):
        
        randomkey = str(random.random()) + '$' + str(random.random())
        randomkeyutf16 = bytes(randomkey, 'utf-16')
        #store this salt, alongside your raw_password
        if saltkey == None:
            salt = hashlib.sha1()    
            salt.update(randomkeyutf16)
            saltkey = salt.hexdigest()[:5]
        preencryptpass = saltkey + '$' + raw_password
        pencryptutf16 = bytes(preencryptpass, 'utf-16')
        hshlb = hashlib.sha1()
        hshlb.update(pencryptutf16)
        hsh = hshlb.hexdigest()
        self.password = '%s$%s' % (saltkey, hsh)

    def check_password(self, raw_password):
        """
        Returns a boolean of whether the raw_password was correct. Handles
        encryption formats behind the scenes.
        """
        
        saltkey, hsh = self.password.split('$')
        preencryptpass = saltkey + '$' + raw_password
        pencryptutf16 = bytes(preencryptpass, 'utf-16')
        hshlb = hashlib.sha1()
        hshlb.update(pencryptutf16)
        hsh2 = hshlb.hexdigest()
        return hsh == hsh2

    def __init__(self, rawpassword):
        self.set_password(rawpassword)
        
rawpass = 'abcdeabcde'
a = encryption(rawpass)
print(a.password)
b = a.check_password('abcdeabcde')
encryptmessage, privatekey = a.encrypt_message('The cat walked home.  The eagle has landed.  The eagle has landed.')
print('encryptmessage: ', encryptmessage)
print('privatekey: ', privatekey)
decryptmessage = a.gen_decrypttables(privatekey, encryptmessage)
print (decryptmessage)

Here pattern predictions in theory or more likely broken up with respect to partitions of the original encrypted letter and privatekey. In theory repetitions of the encrypted letter alongside private key, are less likely repeated in the process of constructing the encrypted message. Thus making for it harder say for pattern recognition on the subset string blocks.

One could further increase the difficulty in patterning of encrypted partition segments on the overall message, by creating another modulus code set that relates to the lengths of the actual partition segments.  Thus instead of having say always 6 unit length segments, one could vary this pattern over the set of the entire message.  Thus a  space ' ' could represent in one instance 'a7'  and in another instance '7ahme'  in another instance, so that frequency distributions and expectations of patterns are further scrambled.  One should be careful in limiting partition lengths of the original encrypted message as nearly single place or two place representations can possibly lead to equivalences with hashed character sets.  Thus the finally I have:

import hashlib
import random
alphanumer = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
              'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x',
              'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',
              'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V',
              'W', 'X', 'Y', 'Z', '1', '2', '3', '4', '5', '6', '7', '8',
              '9', '0', ' ', '.', '!', ',', '@', '#', '$', '%', '^', '&',
              '*', '(', ')', '-', '_', '+', '=', '[', ']', '{', '}', ':',
              ';', "'", '"', '?', '/', '>', '<']

class encryption:
    def generate_moduluslenkeyset(self, epart):
        epartlen = len(epart)
        setrange = range(0, 7)
        rmodlenkeyset = []
        for i in setrange:
            if epartlen > 12:
                ml = random.randint(6,12)
            else:
                ml = random.randint(6, epartlen)
            rmodlenkeyset.append(ml)
        return rmodlenkeyset
        
    
    def generate_moduluskeyset(self, epart, rmodlenkey, moduluskeyset):
        #where epart is the encrypted partition
        epartlen = len(epart)
        mk = random.randint(0,len(epart)-rmodlenkey)
        moduluskeyset +=  str(mk) + ' ' 
        return moduluskeyset

    def generatemsetlen(self, epart):
        #7 set length, although you could extend this if you wanted
        #to.
        setrange = range(0, 7)
        moduluskeyset = ''
        rmodlenkeyset = self.generate_moduluslenkeyset(epart)
        index = 0
        for i in setrange:
            rmodlenkey = rmodlenkeyset[i]
            moduluskeyset = self.generate_moduluskeyset(epart, rmodlenkey,
                                                        moduluskeyset)
            index += 1

        mset = moduluskeyset.split(' ')

        for rmodlenkey in rmodlenkeyset:
            moduluskeyset += str(rmodlenkey) + ' '
            
        return moduluskeyset, mset, rmodlenkeyset 
        
    def generate_privatekey(self):
        
        randomkey = str(random.random()) + '$' + str(random.random())
        randomkeyutf16 = bytes(randomkey, 'utf-16')
        #store this salt, alongside your raw_password
        salt = hashlib.sha1()    
        salt.update(randomkeyutf16)
        saltkey = salt.hexdigest()
        return saltkey

        
    def encrypt_message(self, raw_message):

        
        privatekey = self.generate_privatekey()
        encryptmessage = ''
        capture = False
        lindex = 0
        for letter in raw_message:
            preencryptpart = privatekey + letter
            pencryptutf16 = bytes(preencryptpart, 'utf-16')
            hshlb = hashlib.sha1()
            hshlb.update(pencryptutf16)
            hsh = hshlb.hexdigest()
            if not capture:
                moduluskeyset, mset, rmodlenkeyset = self.generatemsetlen(hsh)
                mset = mset[0:len(mset)-1]
##                print('e mset: ', mset)
                capture = True
                modk = len(mset)
                modl = len(rmodlenkeyset)
            mpos = lindex % modk
            mlpos = lindex % modl
##            print('mset ', mset[mpos])
##            print('mlset', rmodlenkeyset[mlpos])
            mstart = int(mset[mpos])
            mend = int(mset[mpos]) + rmodlenkeyset[mlpos]
            encryptmessage += hsh[mstart:mend]
            lindex += 1
        privatekey += ' ' + moduluskeyset
        print('mset ', mset)
        print('mlset ',rmodlenkeyset)
        
        return encryptmessage, privatekey

    def gen_decrypttables(self, privatekeyg, encryptmessage):
        #this works in a limited context for western latin based scripts
        #you'd have to make sure to import the correct character set for
        #alternate languages.  I'd recommend encrypting into the private
        #key a language code to reduce computation times on character sets.
        keytables = privatekeyg.split(' ')
        privatekey = keytables[0]
##        print('private key: ', privatekey)
        msetmlset = keytables[1:len(keytables)-1]

        setrange = range(0, 7)
        mset = []
        mlset = []
        for i in setrange:
            mset.append(msetmlset[i])
            mlset.append(msetmlset[i + 7])
        print('mset: ', mset)
        print('mlset: ',mlset)
        
        modk = len(mset)
        modl = len(mlset)
        decryptiondict = {}
        for letter in alphanumer:
            keystring = privatekey + letter
            bkeystring = bytes(keystring, 'utf-16')
            hshlb = hashlib.sha1()
            hshlb.update(bkeystring)
            hsh = hshlb.hexdigest()
            decryptiondict[hsh] = letter
        decryptmessage = ''
        count = 0
        
        while len(encryptmessage) > 0:
##            print(decryptmessage)
            mpos = count % modk
            mlpos = count % modl
            mstart = int(mset[mpos])
            mend = int(mset[mpos]) + int(mlset[mlpos])
            for hsh in decryptiondict:
                hshtrunc = hsh[mstart: mend]
                if encryptmessage.find(hshtrunc) == 0:
                    decryptmessage += decryptiondict[hsh]
                    hshlen = len(hshtrunc)
                    encryptmessage = encryptmessage[hshlen:
                                                    len(encryptmessage)]
                    break
            count += 1
            if count > 9999999999999:
                break
            
        return decryptmessage
        
    def set_password(self, raw_password, saltkey = None):
        
        randomkey = str(random.random()) + '$' + str(random.random())
        randomkeyutf16 = bytes(randomkey, 'utf-16')
        #store this salt, alongside your raw_password
        if saltkey == None:
            salt = hashlib.sha1()    
            salt.update(randomkeyutf16)
            saltkey = salt.hexdigest()[:5]
        preencryptpass = saltkey + '$' + raw_password
        pencryptutf16 = bytes(preencryptpass, 'utf-16')
        hshlb = hashlib.sha1()
        hshlb.update(pencryptutf16)
        hsh = hshlb.hexdigest()
        self.password = '%s$%s' % (saltkey, hsh)

    def check_password(self, raw_password):
        """
        Returns a boolean of whether the raw_password was correct. Handles
        encryption formats behind the scenes.
        """
        
        saltkey, hsh = self.password.split('$')
        preencryptpass = saltkey + '$' + raw_password
        pencryptutf16 = bytes(preencryptpass, 'utf-16')
        hshlb = hashlib.sha1()
        hshlb.update(pencryptutf16)
        hsh2 = hshlb.hexdigest()
        return hsh == hsh2

    def __init__(self, rawpassword):
        self.set_password(rawpassword)
        
rawpass = 'abcdeabcde'
a = encryption(rawpass)
print(a.password)
b = a.check_password('abcdeabcde')
encryptmessage, privatekey = a.encrypt_message('The cat walked home.  The eagle has landed.  The eagle has landed.')
print('encryptmessage: ', encryptmessage)
print('privatekey: ', privatekey)
decryptmessage = a.gen_decrypttables(privatekey, encryptmessage)
print (decryptmessage)

This last generation of encryption leads to variable character set partition for character encryption representations.  This further compounds to difficulty in detecting not only frequency of character distributions by way of pattern matching, but also increases difficulty in detecting total character lengths of an inset message.  One could suspect really long messages and files that are encrypted still could have some relative frequency of string encryption occurring with higher proximity.  It seems that even further methods could be devised in randomizing encrypted character partition string distributions, however, or if really interested I am sure you could find advanced research in the area of this topic elsewhere on the internet.

encryptionscript in python

This works off python's native modules so you hadn't need download or install any other modules to operate.

Behavioral modification at its finest...

Seems very up there with bloodletting and other medieval practices...

http://en.wikipedia.org/wiki/Judge_Rotenberg_Educational_Center 

amazing...showing big revenues, lots of lobby, lots of attorney's representing the center supposedly, lots of taxpayer dollars providing the revenues...hmm...sad...

Anyways, one has to wonder in down economy with state's justifying in so many way budget cutting to so many services why there should be a facility potentially costing quite a bit of money with questionable results...I mean apparently in one state the problem is that collective bargaining rights are apparently too much of a costly issue while in another people can't paid well enough to torture kids...if any teacher should lose their jobs, it should likely be at this place for budgetary reasons, and hopefully sooner then later.

Added points here, prisoners can evidently be successfully incarcerated and restrained with respect to their holding, but children neither convicted of any crimes apparently as a 'last hope' can't be provided the same respect with regards to human rights.  After all, this particular facility neither is required mandatory for treatment by any government, local, state, or federal for any student as far as I know, and isn't legally linked in the case to treatment for criminal or felony behavior, nor have the student's to my knowledge ever been convicted generally in their respective past for heinous crimes.  To this end, watched in one case, excerpts of 31 shocks applied through the duration of hours of one particular student that were punished with electric shocks for non compliance in removing a coat.  I mean we are talking a real murderer, a real problem in society there being the social non conformist that he were?  Right?  Another student in the past having died subsequently owing to neglect in medical treatment received, only died as apparently 'teachers' on premise hadn't properly identified that her social non compliance to instructions on premise were owing to a serious medical condition.  She apparently were in significant enough pain owing to her medical condition that she weren't fully responding to the 'teaching' staff, she were tortured.  The next day when she later died, and at least hadn't been given medical treatment appropriately.  This incident took place over a decade ago, and meanwhile the school is still running collecting taxpayer monies with students even testifying before congress, it sounded in defense of the school itself while wearing the electrical shock restraint devices giving such testimony.  Only a in a Mother Jones article a journalist recalled on her last days visiting the premise, that one such electrical shock device were faulty and apparently sending out errant currents, the device hadn't exactly been identified as problem before, but coincidentally appeared to be the case so near to her visiting departure.

Its not that children one would suspect tend to be more powerless, easily suggested and coerced into any number of arrangements that under any other circumstances legal or having been of mature age in making independent decisions, it seems there is illustration of problems often relating to more vulnerable segments of our society, and the proposed solutions in handling such problem.  I'd mention aside from the supposedly more extreme examples of human behavior cited by those one social group of kids attending the school.  Another segment relates to those of physical and mental disabilities.  In this case, voluntary agreements of power of attorney or those of supposedly of better custodial role, neither having lived in the bodies of those subject to pain and torture are empowered to know better and make decisions for the sake of the other's health capacity.  The problem is: if you were of sound mind would you ever likely seek torture as a solution to any number of physiological and neurological health issues that you ever suffered from?  Even if you knew other restraint options applied would seek the infliction of physical pain were the solution to any number of compulsive disorders.  Often times, those that are most vulnerable may be in more difficult positions to adequately understand and communicate with mature understanding their feelings or knowing how to decide for themselves right and wrong in a situation.  Why is that the possibility of torture should still be included if a custodian is permitted some degree of flexibility in error with respect to the welfare of another under their power?  And only this is a for organized as opposed to 'illegal' form of domestic violence in which government, legal systems, and others operating under this are given sanction to impose.  Maybe we think when hearing the word 'disability' we think, 'Oh, yeah, kids that are real problems.'  Maybe we think back to the days of corporal punishment which should seem so steadfast in our culture that we scarcely think of the differences between all other segments of society that have provided little legal recognition to what otherwise would be assault.

The irony is that while someone like I contends with some other segment of society that runs wild often with bad media metaphors...never mind reality and truth here, there is much willful overlooking what should occur elsewhere, much hypocrisy, much double standard.  Only two decades in operation, the place looks worse then the infamy of Alcatraz, and given its per capita profit and expenditures otherwise, looks like any other money pit set up for social political reasons to the better half of biases that have for so long regarding children and the vulnerable.  A grown woman in her thirties having long since left the school needs the device to apparently keep her 'in line' as she needed way back decades ago, and behavioral modification for some may be as temporal as the lack of hunger from a previous meal.  Unfortunately this may relate to some of the problems of this sort of treatment, being forced in line repeatedly may work so long as a drill instructor exists to keep one in line, as to formed life long habits for months or years of pain induced training, as likely maybe a smoker given the same aversion based treatments lasts likely a month or two at times before relapsing as a recidivist into old patterns and ways if not genuinely committed to the sorts of behavioral reform.  Punishment works no doubt, but only when it is constantly applied?

Finally the spiritual dimensions of such a place.  In thirty years if the place were still operational if at that, one suspects so much the tainting of soil and bad karma placed in a place like this where someone inevitably gets burned for the wrong reasons...it could be one of those places later that the paranormal peoples go to for their Halloween gig, only hearing so much the echoes of the past pounding the walls days and nights at times.

Eve marketlogs to ms sql database importer

Having to rework code on this since I lost a backup file and didn't do cloud storage for old query and python files.

To implement importer fully, you should have the pyodbc module.  A c bindings driver that interfaces that provides pythonic interface commands for ms sql.

At the moment its still needs more work, you just need to set up at the program's outset a directory path to Eve market logs folder.  There are three main table types where data is sent: reactionmaterialsindex, itemsindex, and myorders.  Also you'll need to make sure that you enter correctly your sql server database information, alongside user id, password inputted.  I may eventually get around to creating a windows based executable but for now you'll need to do this manually.

Dependencies:
python 2.7 + (works with 3.2)
pyodbc
MS Sql database2012 (hadn't checked other versions)
Eve fansite kit
Evesqlimporterscript

Windows installation procedure:

Download and install the above program dependencies.  You'll obviously need to install python2.7 or 3.2 before you install pyodbc.  Just grab the appropriate pyodbc module based upon your system and python version.  Then install based upon your system, the MS Sql database 2012 with tools option.  Then grab the fansite kit (just need Inferno_1.0_70633_db.zip).

Once you've installed MS Sql 2012, run Microsoft SQL Server Management Studio.  Firstly I recommend in Object Explorer (left hand side of Studio window) right mouse button clicking on 'Databases' , here select 'New Databases' and then input in the Database field name 'Eve'.  Press Okay.  Next for the pyodbc driver you'll probably want to have SQL authentication enabled, so you'll need to enable this authentication by selecting in the Object Explorer, your Server Name, mine is for example, CHRISTOPHER-PC\SQLEXPRESS, it is the top most object item in the explorer's data tree structure.  Right mouse click on this, and select 'Properties'.  Then under 'Select a page'  (left hand side of pop up dialog), go ahead and select, 'Sql Server and Windows Authentication Mode'.
Next you can create your Sql authentication based user id, by selecting in Object Explorer, YourServerName > Security > Logins.  Right click on 'Logins' and select 'New Login'.  Under  'general' (Select a page tab) of your 'New Login' dialog, select 'Sql Server Authentication'.  Then enter your 'Login name' in the provided field, make note of this, you'll need to add this information to the python script that I have furnished.  While there are many options under the 'Select a page' (left side of dialog, under 'Server Roles', 'User mapping', and 'Securables'.  By default granting permission and check marking all fields for 'Server Roles' and selecting your 'Eve' database, should grant you necessary permissions for your userid.  Make sure you have entered a password.  This has a minimum characters requirement default, so make sure this is at 15 alpanums in length (at least I haven't been able to input anything less).

Next using a favorite text editor or using Server management Studio direct, copy and paste the ebs_DATADUMP sql query data into a new query.  On Server Management Studio, you can select New Query, making sure that YourServerName > Databases > Eve is selected, and then paste the sql query code into the new query window that should be populated, or open the ebs_DATADUMP file directly from Server Management Studio.  Then press execute.  If all goes well should have new tables (with no data) furnished on your new Eve database.

Next you'll need to restore the backup database file for Eve.  You can do this from Object Explorer, selecting YourServerName > Databases  and right clicking Databases, you can select 'Restore Database..'  I've found that owing to the silliness of the file finder dialog here to have the DATADUMP201205212008.bak (or related file depending on what version of Eve is presently current) having this setup in a convenient locations like 'C:\Users' by default.  By default under the 'general' tab, select under 'Source' the 'Device' radio button.  To this right of the now blank field under this radio button, select the '...' button.  To the right of the presently blank 'Backup media' text field, select 'Add'.  Then go to your source directory for the .bak file using the pop up directory tree explorer.  Your backup file should be present in the file field.  Select this, press Okay, and then push Okay again on the 'Select Backup Devices' dialog.  You should now have the .bak file showing on your 'Restore Database' dialog now.  Under 'Select a page' of the 'Restore Database' dialog, you should select 'options'.  Then select 'Overwrite the existing database (WITH REPLACE)' and then select 'Preserve the replication settings (WITH KEEP_REPLICATION)'.  Push 'Okay' on 'Restore Database' dialog here.  If all goes well you should see a confirmation that your database has been restored.  You can check for data in this database, by selecting in Object Explorer,

YourServerName > Databases > Eve > Tables and then selecting any one of the given 'dbo.' tables shown.  You can right click on the table, and select 'Select top 1000 rows' .  This will automatically generate a query  for you populating database information for the given table.  I highly recommend that you have some background in SQL, so that you can create your own custom queries likewise.  Any decent book like SQL Visual Quick Start guide by Peach Pit Press publication for instance, can be an excellent start for learning some basics to query language programming.  I also recommend some requisite knowledge of python here to aid you likewise.

 Now that you have data in the database, using your typical market export methods in game, and presuming you have something in marketlogs folder to import into the database.  You'll just need to furnish your server info, userid, and password at the 'cnxn' variable of the Evesqlimporter script. And put the directory path to your Eve market logs folder at the 'dirpath' variable.  After running the script you may need to close down the existing sql server socket connection manually to populate say added data in Server Management Studio.  Apparently I've run into difficulties with more then one server socket connection utilizing pyodbc and server management studio at the same time.  In this case, just close out the idle terminal, not the script.

Finally while I have a batch file move script, to place a file in a given backup destination, I haven't implemented this.  You could do this on your own if you like, or move the files to a backup destination other then your present directory.  I may add for convenience sake later a file sync script, that records and tracks a list of uploaded files into the given database, thus avoiding having to do anything further other then making sure that files added to the folder or import merged into the database once synchronization is enabled.

Updates 6/11/12:

Materials Reaction Average indices python script

 EVE MS SQL database market tool works in conjunction with another importer I had designed
EVEsqlimporter.
For the given reaction materials indices table, these are materials associated with the EVE reaction materials table, an average price is computed for the given top 10 high and low orders of the given material's market buy sell orders, also filter's for order movement.  Will make updates later differentiating by region, and may increase volume movement threshold so as to throw out obvious order's that were obviously well off from a movement price average.  At present this also discards old orders beyond 90 days or those of elapsed duration, based upon both date of order input and batch date entry of orders furnished at such time.

Computed average indices are imported to the reactionmaterialsavgindex table.  This table is automatically created here for you.  Just be sure to refresh your database through Management Studio
if checking for verification.

Updates 6/12/12:
 Materials Reaction Average indices python script

-Computed averages are now differentiated by regionID (regional markets), key added for this on table.  Also added standard deviation column.  Provided filters for consideration of price order.  Where, for instance,if standard deviation / computed mean > PCNT_STD_DV in script, then maximal price order is discarded and recursively averages are recomputed until standard deviation and mean fall within the given relation threshold provided, yet another tool aimed at filtering obviously high ball or low balled orders probabilistically least likely to filled in terms of buy or sell respectively.  Added DISTINCT on query to ensure repetitions of orderID are not included (because of orders, for instance, appearing over any number of batch entry dates).

Generate items reprocessing estimated value python script

Lazy tonight will finish this up tomorrow for fetchall() or if you download feel free to update, just
need to fetch the rows and grab tablevalues. Will probably update this to write to dictionary.
At some point probably going to develop a Gui for all this so that a table populates this way.

If you are new to SQL I used a table JOIN method to cross link the data from various tables.
I also used an aggregate function SUM and had to use an aggregate GROUP BY clause so as
to be able to ensure that aggregate SUM s were applied by typeID and not every row value
irrespective of typeID.  Also I had to link materialtypeID (this is the reprocessing material ID...for example, Tritanium, Mexallon, and so forth), with the computed average indices table for such materialIDs, once having done this, and having computed the reprocessing/manufacture materialIDs quantities with the average pricing, I summed these values grouped by itemID.

6/13/12 updates:

fixed pricing list generate sql query line...should work now in script.  Also, added regionID filter
when pulling regional reprocessing/manufacturing materials.  Also creates an 'items pricing' txt file
which includes line itemization of both item name and pricing estimation.  

6/16/12 updates:

At present sort of exploring ideas here.  Working on graphical ends, some thoughts as of current.  Created a data synchronizations tab, sort of inspired by cloud type synchronization ideas here, but at the moment have kept this restricted to user interface toggling.  May update synchronizations in tracking market log files, also providing a list of loaded market log files?!

Some basic ideas that I'd like to work on:

Creating an Items Marketplace explorer.  This is not unlike a folder/file directory explorer, but instead uses the data tree found in Eve's market.  Key table for doing this is the marketplace groups table.  So my idea here for code implementation is fairly simple:  simple query for data export, and use a recursive function for building data tree.  Then to create an data import implement for data tree widgets (as I've seen commonly referenced in Gui languages).   The purpose of the explorer should be simple enough, like being able to create customizable query filter searches for any list of items, alongside, providing an additional search query tool.  I've seen this at least in other related type apps likewise.  The marketplace group table should make construction of data trees very easy once the recursive function is found for linking groupID and parentID s here.  In SQL you could do something like joins, or inner joins, and some SQL platforms offer recursion (think MS SQL offers this) while some are non recursive.  SQL varies from other programming languages in terms of feel especially if you are used to the sorts of procedural flexibilities that allowed you to loop in more abstract way over a given dynamic range of data.  If you weren't so well versed in SQL, this can be tricky.  More likely the SQL solution would be a fixed set of table joins that might cover many trees, but not all here.  Exporting data to a procedural language like Python, Java, C++ and so forth allows you on the other hand to create tree probing methods (recursively, or abstractly enough here) that could be more difficult to program in SQL (supposedly SQL means Set Query Language by the way).

Would also like to implement profiles that a user could save in so far as session work.  Basically if a user want a customizable search query, the data that they were most interested in could be easily populated yielding an number of items at once...without having to tab through query drop down boxes, or key search for any of these items in a big list otherwise. 

A open query box popup (simple), so that a user can input a custom query for execution...

Query fields for refined search (sort of open in thought and related direction here)...

Possible future things:  Manufacturing calcs and the like.   I once tried a fitting calcs idea, but I never completed it...sort of tried to create this fittings optimization algorithm...might try working on it again, depending.

Sticking with python's already packaged Tk here for gui writing, one since in window's it looks decent enough, and two I hadn't need go through a bunch of binding layer's to program a Java gui that basically looks about the same...Java seems like it could be nicer on some ends, but basically Tk provides enough for this project anyways, and then if you wanted to use python to program a Java app, you are actually running a python 2.5 version, or you could technically run using exec() type commands in higher version's of python Jython program's but not sure if it were at all possible to pass instanced data under jython compiler execution.  Gtk in windows isn't good by the way, or at least they have old windows binaries going back to 2009 :)  ...Tk is more up to date, and Tk is easy to work in the non visual approach, especially in Python.  Seems a little less sophisticated (but actually a little easier) then Gtk but decent enough.  Tk isn't so nice looking in Linux by the way (at least Ubuntu...so I highly recommend Gtk or other alternatives for gui app development)..data tree widget's in Gtk are a bit more technical on the implementation side of Gtk relative to Tk also. :)  
 

Space colonization report card

     I know you read lots of fiction about people that live in under ground bunkers for decades and what not.  The problem is that as far as I know generally people don't live this way, and generally haven't been much successful at it.  I would refrain at the point of austere monks that have by way having volunteered themselves for the sake of their own religious practices and observance in more extreme conditions of environmental deprivations, but even then one could neither describe their conditions as completely closed in the environmental sense, or that others would supplement their environment in some way so that their own existence should be feasible.  Prisoners require this as would generally most human beings one could imagine.  Practically speaking, while at any given time, one could see more so the self evident nature of human existence so adapted to the ecological habitats of this planet while, however, diverse these should be, placing some practical limit on the nature of one's existence elsewhere.  If one were to attribute the successive generations of crop growths having produced any manner of vital nutrients in sustaining life here, while it could be assumed in the short term could be related to any number of basic nutrients, one could argue as to the nature of any number of micro nutrients, micro organisms and problems that could arise therein in simplified closed systems  which could effect biological engineering projects, or at least it could also be problematic given limitation of space and only that much greater dependency on the productivity of that space for one's physical livelihood that less room given for problems given the lack of biological evolution typified for the same productive use of such organisms in a context

And then contrary to movie and fiction, I'd challenge someone to produce evidence that underground fish farms really do work and last for decades.  Or challenge someone to grow sustainable crops for food stuff agricultural productions for decades in artificial environments without so much interactivity from an outside source.  Yes there may be underground hydroponic farms for illicit drug trade, but I would offer even here systems are not completely closed.

Most likely first colonists would have to engage in re supply logistics, and might use in a limited context self produced goods that they could in part live off of, but likely even here, cramped space and quarters, and sterile white looking interior environments could be problematic to one's psychology over a length of time.  While you might be able to go outside with the assistance of any life supporting mechanism, unless materials technologies were advanced at the moment, bulky, cumbersome and unseemly gear generally limiting physical movement with respect to long term living circumstances could present yet added difficulties to living.  What should seem small trivial issues here in the terrestrial sense, could be problematic over time in handling from a psychological standpoint.  I'd suggest for those interested to read any number of real life longer term expeditions and voyages, that weren't so successful or even if having been successful what experiences might be typified where environmental conditions were limiting to the degree of one's existence or where psychological stresses could be likely found.  Unfortunately, if in the terrestrial sense one should find a given environmental extreme, Mars at best in the thermal sense comes closest to freezing a few inches in elevation off its surface while elsewhere in its atmosphere in their 'summer', thermal extremes abound relatively speaking which should make our polar climates look like the tropics, or if you considered the major differences in atmospheric pressure, your life is generally consigned to specially designed habitats that are foreign to mars itself.  Sure NASA sent people up into the Arctic extremes to study in a limited context habitation possibilities or in the case of to test robotic equipment, for example, or maybe one could apply studies from known research outposts, but entire colonies of people living elsewhere for life?  Most outposts aren't constructed on the premise of making a colony a place of habitation for life when it weren't conducive to life one could imagine for a number of reasons.  Cost and feasibility just to name a few.  Living where breathable air shouldn't exist may have greater associated cost mechanisms.  The more complex machinery adds something in principle:  that much more that could go wrong, and the more complex the assistance of mechanisms aiding life that couldn't be reproduced given the differences in native ecology and environments only add to these problems.

     Sure people want to be inspired by the next generation race to something, or live on a dream, but there are many steps involved in the actuation of dreams like this.  If you had a litany of studies and research touting proof of your claims, that you could do something and practically do it successful, I'd give you a pat on the back and blessing to go with it, but otherwise...  .I'd say you'd better sticking some of these colonist wannabes up in the Arctic for years and see how well things go in their little life capsules up there?

     Challenging lifelines in impractical and highly untested ways could be likened to thrill seeking, a form of gambling, or yet another example of ignorance or arrogance otherwise for much that could be taken for granted.  One thing lacking in human civilisations at times that one at times might sense, outside of the branches of sciences which more readily could appreciate the longitudinal sense and scale of endeavours,  the use of 'colony' in times historically has been marked with much tragedy with respect to any number of ill prepared groups of humans arriving and being highly interdependent for their given life lines actually on their respective host environment.  Then I have thought often personally in some respect to desire when it has been impractical, and it seems more likely an immaturity born in our own condition, if only we have spent little time challenging and knowing the ramifications of challenging our own biological life lines, we should likely have little practical experience doing so, and often times practical in experience more so relates to the nature of what un forseen that should happen.  A connection to this problem of practical experience versus theory in mind of possibilities are that often times, while our minds especially in the collective sense are powerful to imagine all sorts of contingencies respecting problems to survival in extreme and harsh environments, practical experience ranks the highest.  This mean being armed with empirical sets of data and studies, and having the practical wisdom and experience of having lived in conditions that provide use and merit of the practices that have been developed for living in such environments.  Imagination only goes so far with respect to our extremely complex biology and the complexity of our physical universe as a whole, and the same can be said of intelligence, and dreams likewise.     

The Case In The System Pur - short story

The Case in the System Pur (epub)
The Case in the System Pur (mobi)
The Case in the System Pur (pdf)

Prologue

A great gravity tidal wave pulled at the seas of __, caused the amassing of a new mountain, and spurred so much the eruptions of many places thought once dormant. A century would pass before the climate of __ appeared to resemble somewhat a place that had been known. The origins of the destructive wave weren't exactly well explained, seemed to surface as likely as it had come and gone. Some conjectured that it were some dark hidden matter that must have sojourned the recesses of deep space, only its presence were known in the vicinity of some galactic radial neighborhood. At least it were thought not originating in presence to the current galaxy itself because astronomers on the planet ___m noticed nothing of local orbital variations. At least not until a variation were spotted ten years prior in another star system. The effects to the star could be seen thereafter as though it star were turned inside out with dazzling if not worrisome releases of energy, but this were the least of problems it should seem. The worst yet were that the entire system hadn't been perturbed from its present relative orbital motion. Of course, this weren't so dramatic as in the star literally traveling quite clearly through the night sky as a planet should seem relative the other stars in a local system, but shifting in different directions that overtime conspicuously revealed itself.

Then a rush were on by astronomers, scientists, and engineers to verify by way of historically recorded astrometric data that might have revealed anything of a similar pattern that might have evaded notice up until now. Consistent survey sweeps of stars were recorded across broad arc swaths. In centuries passing, such data were kept. While anomalies could be spotted if only found by resolution of the math, machine and instrumentation available at such a time, but oddly enough nothing could be found clearly in previous data. Many stars had wobbles of one sort or another, and apparent resolution of the wobbles themselves were only magnified in time, so that the wobbles would appear more like many wobbles based upon its own local gravitational exertions. This seemed to be consistently spotted of data at least in recorded astrometrics, but not only a strange movement in the start of system __ should occur that hadn't previously known, but its appeared to be moving unusually enough in time lapse.

While alarm had been raised, then general consensus agreed little prediction could be made. Only this variation oddly enough found in a star system some ten light years away could tell little of the trajectory of some unseen dark matter if it had existed, and what matter remaining at levels too level for discernible detection because of weaker luminosities and internal radiance between such star and the present system of __m should only make difficult subsequent detections. Nothing having passed up until then at that time ten years prior when detection were first made, indicated that any dark matter neither moved at the speed of light, or indicated if a course were in some other direction altogether. Subsequent planning and contingencies by governments themselves were limited.

-2-

The mysteriousness of the detected gravitational signature however resurfaced only later, this time approximately ten years past. Nothing much could be done. The world should have unfolded on itself and most if not all life on such planet cease to be, but this weren't the case. From one moment all life suddenly reappeared in what should have appeared to be a different planet but were their own. Their star appeared slightly compressed, more so eccentric in shape and having appeared slightly larger larger in their sky then they had previously known. Few might have guessed where they might have lived before. Except that now as if having awoken from a sleep, they wandered about their once cities that appeared differently arranged to them. Cinder cones occupying farmer's fields in some cases, and houses perched about the many ledges of new mountains. Impossible if not impassable roads set about inclining the steepest valley walls in some cases of great metropolis that appeared blanketing portions of skyline. In other cases, seas completed invaded cities. The odd aspect of all this were not only had people not died as appeared to be the case for much of life, but that they too were somehow different. At least many had the feeling that they somehow knew something about their new world. Like how to get to the grocer when the grocer were several thousand feet overhead, and seemingly all but impasse should abound, or how to retrieve a car that were now resting near the bottom of a lake bed. Solutions of the simplest sort might have been simply riding the mile high escalator, finding one shopping basket where it were parked in the usual grocers entrance, only the store it seems were so vastly huge and stretched alongside the store shelves. Strangely enough nothing of the familiar constructions of uniformity no longer existed. Designers and architects sometimes wept, while others laughed. Guards stood idly by in shock as the prisoners of a local prison, once having roamed so often in their kept spaces, walked through the stretched and elongated chain links of a security fence, and a prime minister found it exceedingly difficult gaining entrance to quite small residence which appeared miniature relatively so, a passerby happenstance landed an errant foot in the wrong place, and the dwelling couldn't be saved.

The other strangest aspect of all this were that people walking through ordinary space could be relatively sensed in more obvious ways growing and shrinking without warning, or that they themselves when asked to run a straight line would suddenly find themselves oddly walking or running some curved manner. Story headlines broke that the space time curvatures were neither uniform, and something of unseen matter were present causing spatial distortions, but why were they alive at all if this should be the case, and even stranger spatial distortions weren't causing noticeable exertions on surrounding matter? At least this some expected that if matter itself were causing distortions in surrounding space causing noticeable dilations, it were expected that gravitation as a common physical manifestation should be expected, but this weren't the case.

Only oddly enough at times given the right locality, it seems the star overhead appeared to be dancing around in the sky, but in other places, the star were where it should normally have been expected.

Some recalled with religious fervor miracles while others weren't exactly sure. Only when it appeared that events, however, so strange and exotic that might have evoked the sentiment of so many emotions and passions alike, now everything should seem ordinary about such events even if rightly so much the world hadn't adapted.

-3-

The Story

Jom were awakened by screaming the morning when the 'wave' had passed. The windows in his small two room apartment were closed which alarmed him so much more. He hadn't heard the warnings given otherwise which had sent people scurrying into the streets, but even then he hadn't seen this exactly as his window shades were drawn only because he often would wake in the earlier morning hours unable to sleep until dawn came. He slowly rolled himself out of bed. He thought he might check the morning news on his console as he normally would and chew on lipid bearing toast, and while no one in years had sent him any personal emails generally that couldn't be categorized as solicitations of one sort or another, he likely found some personal interests checking even this, or as he would have well reasoned, just in case, years in passing someone special to him should suddenly surface that were critical in changing his life directions. Jom had stumbled forth to the bathroom first, his mouth were dry, and he almost fell through to the great chasm that instead appeared to him. He held on to the door between his bedroom and bathroom that were so much the difference between life and death. Somewhere Jom heard his cat calling, and soon his cat, Rummer appeared to him walking the length of what appeared to be a long stretching and twisted corridor. The cat appeared like an ant winding itself along the stalk of a tree branch circling round and round.

Jom had placed his foot back onto the ledge representing at present his bedroom. More carefully studying his room, the windows appeared mostly the same except for one which were slightly bent. His shoes, however, by his bedside only appeared enormous, and then swept around this area it were as if a magnifying lens had been focused about his bedside where each granular hair of shag carpeting pile could be seen. Jom rushed to a nearby window. Drawing the blinds, now he could see more clearly the chaos of the city that once were. Buildings appeared oddly angled, alongside windows, and warped mountains with people streaming alongside every ridge. He could see some having fallen while others appeared impossibly walking the vertical walls. Jom almost had forgotten that the great mountain before him were once the central business district of the town __.

In the nearby apartments people were shouting and screaming, while another woman had fashioned a rope made of cloth sheets segments tied end for end. Only it were so hard to tell where the ground begun, and relatively speaking the cloth rope seemed to be dangling for what should seem the reflected images found between mirrors, seemingly perspectively almost vanishing from site, but a pocket of ground could more clearly be seen in one other location which were situated several apartments over.

It appeared that the ledge above this location were vacant.

Jom had stepped outside the window to the nearest of a the apartment building exterior facade, a small ledge provided foothold for his toes while he held in his right hand the window frame. He remembered that he were four floors up, which meant they must have been forty feet up. If only he could reach the woman he could clearly be of better help trying to figure something out. When he had provided himself as much distance as possible stretched between his open window with outstretched hand, he jumped sideways, catching his hands on the rails of a neighboring balcony. He could see his neighbor more clearly now except for the appearance of a stationary mirror of the room in the space between her and the window, and the shape of the room passing through this mirror were disfigured and warped. Jom hadn't given any regard to this only attempting to reach here now, but as he passed into the mirror, he could see her image suddenly becoming more distant, as though she were now miles away suddenly. He stretched his arms which now extended so far that he could barely see his hands. Jom then stepped out of the spherical aberration. He couldn't see her anymore. 'Where had she gone?'

He recognized the room and what should seem her kitchen disfigured beyond recognition. He yelled her name. A faint response seemed to come from all around. As if she literally migrated inside his head at the moment, yet he couldn't see her, but he could sense her or feel her somehow so near to him. He had the thought of the made rope. He yelled out loud that he were going to attempt to make an attempt for the next door balcony, and that she should follow him.

Jom edged his way carefully to the ledge of the next door apartment. Only this were much better. He called out his neighbor's name. She weren't there. He could feel the rope now bearing mass. Then the rope had gone slack. He lowered the rope over the ledge. He could see that it were near to the bottom, at least close enough so that the drop should seem reasonable from his vantage, but hesitantly Jom thought alternately that some a door in this apartment room might provide easier escape. Jom threw a nearby balcony chair into the balcony's entrance which caused the window to shake violently for a brief moment but not break. Then he noticed smoke billowing from a nearby open window.

Jom covered his shirt with his face. His eyes were burning he could scarcely see in front him inside the foreign room. He hadn't known this neighbor so well, who he were, another anonymous person that he might have generally passed once or twice in the hallway. No one replied to his cries now, however. He could scarcely make out the lines of a bed. In it he could sense a stationary figure. Jom rushed to the person laying there. A billow of roiling hot smoke filled his lungs and burned his eyes so much that he could scarcely stand while nothing further could be done to revive the man whom appeared motionless through all commotion. Stumbling back to the window that he entered, Jom climbed back on the ledge.

-4-

Jom landed somewhere near a curb's edge on a street now flooded ankle high with water while people on foot wandered past. A group of business men passed near Jom chatting amongst themselves. Having stopped only several feet from a shop front which were situated at the group's feet, the group remained poised in some casual social conservation that could have been expected during some luncheon hour. Their voices vanished suddenly although Jom could see them clearly laughing. One of them reached for the door at their feet leading to a smoke shop, and as quickly as the departed. Passerby s clutching casually bags as to be expected on any other day prior were as likely amongst other's whom wore disoriented and dazed expressions. The buildings seemed swirling in Jom's mind. Compulsively he sought nearby steps of a narrow corridor between nearby buildings.

A child tugged at Jom's the sleeve of his shirt. Jom must have been asleep, hadn't remembered that he were perched at the foot of the steps like sometimes other city wanderer's that hadn't a place to sleep for the night. The child appeared to be mumbling words at him. At least then Jom hadn't comprehended much of anything. The buildings themselves appeared less obviously moving in mind or from what he could tell at times literally, and his stomach weren't churning as before.

'Mira wants you to meet __ __ Column. __ __waiting __,' the child spoke this time more audibly.

Who were Mira anyways? Jom thought, and this aside he hadn't been there in years. At least not since he moved from the old neighborhood where it more likely a place of convenience where often times he might have likely found himself in a dark corner with drink in hand considering so much how life could only be more perfect if time were stretched then from now until the next day's eternity, and he were disfavored anyways as a local patron sometime ago when he were in one his more paranoid and hostile bouts. Most had forgiven him for that, he figured, but some of the regulars just hadn't let it go.

The road to the Column had always been a winding destination before and added to this Jom were walking like others precariously on glass and the trim molding of buildings which shifted and swayed by the weight of so many people on foot would provide. It weren't as if the buildings collapsed. At least there were still the building whole that existed beneath. Jom could see the bodies of people that had fallen through glass at times only beneath in piles at the back of the buildings amongst so much that might have existed on shelves or racks piled and scattered around. Someone already thought to lay planks of wood between the openings of doorways in these cases.

Convoys of military haulers appeared dispersed at the Hish intersection although at times largely empty. More often Jom could hear screaming coming from nearby buildings, alongside automatic gunfire. When the soldiers appeared, they were pushing people bound with their wrists behind their backs and blindfolded through streams of people on either side of the row of the military trucks parked. The soldier's faces were as dark as the night, as were the the captive peoples in their possession that were placed at times in the back of trucks while the soldiers at times glanced around having taken little notice of any other groups of people around.

A crew of peoples had cordoned off streets in what appeared to be scenes for a movie on a nearby street. Artificial leaves were strewn about what appeared to be a stone laid arched wall covered in vines that shouldn't have existed were it stood, only Jom imagined that the camera angles would noticeably disguise what should exist beyond the frame itself, and then Jom could see the crew closer as they exhaled. The frost of their breaths could be seen, not like him or others around present, but as if they existed in some other space and time.

Amidst the commotion only few seemed to notice the others. Where at any moment pools of blood, tears and a stench ridden atmosphere of chaos. Suddenly celebrations were amassed in the streets. The military s convoy gone, blossom's on trees, the smell of fresh warm rain laden in the air. Jom lost track of the advance and retreat of shadows, darkness, and passing of light. The Column were still there as usual half buried into the earth and as he imagined likely a cool as the subterranean stones would emit, its shape and form appearing to Jom as he remembered it. Summer sweltering and ushering him onward.

Medics were tending bodies inside the Column's bar. Mira, the woman Jom had never met, must have noticed him, and waived to him calling his name so loudly that her voice should seem as if next to his ear. Her makeup were slightly melting so much when he met her, he thought instinctively replying, 'Is this too much here?'

'Oh, no,' she replied brushing streaming tears of mascara aside. Jom felt then a hypodermic needle thrust into his arm, while someone were yelling and screaming to another medic. Jom calmly insisted he weren't fine. Mira insisted otherwise.

-5-

Several times Jom felt his head as though it were like a stone being struck against another solid object. Several times more, the memory of Mira gone, and a previous history in life to go with this. Dark matter coalesced all around, more acutely the city had become more steeply walled in upon itself, and so many fragments and memories in time resurfacing in a chaos of time that should scarcely exist. Ghosts were reappearing all around. He couldn't sense who or what he were anymore let alone the place that he lived in.

Then the world reappeared in a more familiar form, albeit much wrecked, but there.