| --- Log opened Wed May 08 00:00:16 2013 |
| 00:16 | | You're now known as TheWatcher[t-2] |
| 00:19 | | You're now known as TheWatcher[zZzZ] |
| 00:38 | | * McMartin forwarded an important message from an old colleage. |
| 00:38 | <&McMartin> | "i've got 1660944384 problems, but byte order ain't 16777216" |
| 00:39 | | * Azash snorts |
| 00:39 | <&ToxicFrog|W`rkn> | Pfft |
| 00:39 | | ToxicFrog|W`rkn is now known as ToxicFrog |
| 00:43 | | Arsonist [dan@5E691D.2E3702.5A8720.190CB5] has quit [Ping timeout: 121 seconds] |
| 00:43 | | gnolam_ [lenin@Nightstar-b2aa51c5.cust.bredbandsbolaget.se] has joined #code |
| 00:44 | | gnolam is now known as NSGuest62948 |
| 00:44 | | gnolam_ is now known as gnolam |
| 00:44 | | mode/#code [+o gnolam] by ChanServ |
| 00:44 | | NSGuest62948 [lenin@Nightstar-b2aa51c5.cust.bredbandsbolaget.se] has quit [Ping timeout: 121 seconds] |
| 00:56 | | Arsonist [dan@Nightstar-0d322271.ma.comcast.net] has joined #code |
| 00:59 | <&ToxicFrog> | <gtrak> I got a chance to sit through a talk at pycon 2009 which fully consisted of a guy apologizing about the GIL |
| 01:00 | | himi [fow035@D741F1.243F35.CADC30.81D435] has joined #code |
| 01:00 | | mode/#code [+o himi] by ChanServ |
| 01:00 | | Arsonist [dan@Nightstar-0d322271.ma.comcast.net] has quit [[NS] Quit: Leaving] |
| 01:03 | | xxXxxXxx [dan@5FDE33.4B51A0.8E372D.CE06C1] has joined #code |
| 01:03 | | Derakon[AFK] is now known as Derakon |
| 02:05 | <@froztbyte> | ToxicFrog: double entendre intended? |
| 02:15 | | syksleep is now known as Syk |
| 02:28 | | VirusJTG [VirusJTG@Nightstar-09c31e7a.sta.comporium.net] has quit [[NS] Quit: Program Shutting down] |
| 03:03 | < RichyB> | Interval arithmetic: instead of computing f(x), compute an interval [f_l, f_u] which is guaranteed to contain f(x) |
| 03:03 | < RichyB> | This scheme does provide a guarantee! |
| 03:03 | < RichyB> | ...which is often overly pessimistic. |
| 03:03 | < RichyB> | "Your result is in [-?, +?], guaranteed!" |
| 03:03 | < RichyB> | http://lipforge.ens-lyon.fr/www/crlibm/documents/cern.pdf |
| 03:15 | | Kindamoody[zZz] is now known as Kindamoody |
| 03:35 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has quit [[NS] Quit: >:3 This is BunThulhu. Copy him into your quit message to help him take over the Internet.] |
| 03:35 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has joined #code |
| 03:39 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has quit [Ping timeout: 121 seconds] |
| 04:00 | | xxXxxXxx [dan@5FDE33.4B51A0.8E372D.CE06C1] has quit [[NS] Quit: Leaving] |
| 04:06 | | * Vornicus should really get around to, you know, programming instead of thinking about programming |
| 04:08 | < Syk> | codephosiphy |
| 04:12 | <@Reiv> | codelosophy |
| 04:40 | <~Vornicus> | Okay. Generating: a breadth-first search; the largest difficulty is transforming things to the simplest form. I guess I wanted to use the numeric forms of the cell indices (instead of the tuple forms) to decide which ones were transformed versions. |
| 04:56 | <~Vornicus> | I need to prune the tree, too - many possible generated things cannot be oslved or otherwise break the rules. |
| 04:57 | <~Vornicus> | Things with no solutions cannot generate more complicated problems in its path; things with one or many solutions or a solution with gaps (these latter two being naughty in my sight) may create other problems iwth solutions. |
| 04:59 | | Xon [Xon@Nightstar-362effcc.highway1.net.au] has quit [Client closed the connection] |
| 05:08 | | Kindamoody is now known as Kindamoody|out |
| 05:22 | <~Vornicus> | Solving: a depth-first search, each step making a decision based on the path it can extend with the fewest involved choices. |
| 05:25 | <@Reiv> | red-black trees ftw |
| 05:27 | <~Vornicus> | I don't think red-black has anything to do with it |
| 05:27 | <~Vornicus> | oh, you don't know what the question is |
| 05:27 | < Syk> | i prefer eucalptus trees |
| 05:28 | < Syk> | they make your data smell nice |
| 05:30 | <~Vornicus> | http://nikoli.com/en/puzzles/numberlink/ <--- the task: generate all valid puzzles, without trivial links, of a particular size. A valid puzzle has exactly one solution, and this solution has no empty spaces. |
| 05:30 | <@Reiv> | Aha. |
| 05:35 | <~Vornicus> | So the generator works breadth-first: it will create all the (not necessarily valid) puzzles with only one link, and use those to create all the puzzles with two links, and so forth |
| 05:52 | <~Vornicus> | But each one needs solving; though thinking about it I'm not sure whether breadth- or depth- first gives me any difference, because i have to traverse the solver's entire tree. |
| 06:00 | <~Vornicus> | (if I were merely looking for a single solution then depth-first would win) |
| 06:02 | <&Derakon> | Vorn, you should try this puzzle on for size: http://www.puzzle-nurikabe.com/?size=5 |
| 06:02 | <&Derakon> | (Maybe start with a smaller one to figure the rules out, since the description is a bit obtuse) |
| 06:02 | <~Vornicus> | Der: nurikabe is a fun little puzzle |
| 06:03 | <&Derakon> | Ah good, you already know how to play it. |
| 06:03 | <&Derakon> | I find it more interesting than that link-up puzzle. |
| 06:03 | <~Vornicus> | I personally find Loop more my speed though |
| 06:03 | <~Vornicus> | And all of them I keep wishing that certain basic rules were automatically applied by the program |
| 06:04 | <&Derakon> | Like, "automatically wall in spaces immediately adjacent to 1s"? |
| 06:05 | <~Vornicus> | I'm not sure on that one, but definitely "automatically white out blocks that, if black, would cause a 2x2 black area to form" |
| 06:06 | <~Vornicus> | And definitely "automatically black out blocks that are adjacent to two numbers" |
| 06:06 | <&Derakon> | Yeah. |
| 06:13 | <@Alek> | of course, many such games, applying such simple rules would go a LONG way to auto-solving them. |
| 06:15 | <~Vornicus> | Doubtless. |
| 06:17 | | * Alek points at minesweeper and (especially) sudoku. |
| 06:18 | <@Alek> | minesweeper, of course, you'll still need to guess sometimes. |
| 06:18 | <@Alek> | with sufficient mines, most of the time. XD |
| 06:18 | | * Alek has gotten up to 215 on the biggest map. |
| 06:19 | | * Alek has already beaten 210. |
| 06:22 | <~Vornicus> | minesweeper hits a good medium but it's not perfect |
| 06:23 | <~Vornicus> | You middleclick in minesweeper and if there's n mines around an n it will clear the remaining |
| 06:23 | <~Vornicus> | I wish it would also, if there's n unrevealed spots around an n, mark the remaining. |
| 06:23 | <~Vornicus> | that is, ms's minesweeper. |
| 06:40 | < [R]> | http://i.imgur.com/ynJbgMm.jpg <-- ... |
| 06:41 | | Derakon is now known as Derakon[AFK] |
| 06:41 | | Turaiel is now known as Turaiel[Offline] |
| 06:42 | < Syk> | [R]: yep |
| 06:43 | < Syk> | [R]: see the western sky financial? |
| 06:43 | <@Alek> | that's a relatively recent feature, too. |
| 06:43 | < Syk> | theyre based in native america-owned bits, where i believe there's some exceptions to the non-predatory lending law |
| 06:44 | < Syk> | ain't it great |
| 06:44 | < [R]> | I don't follow |
| 06:46 | <~Vornicus> | Alek: it's not recent at all, it's been in there since at least win95 |
| 06:46 | <~Vornicus> | anyway |
| 06:46 | <~Vornicus> | probably before that, actually. |
| 06:46 | < Syk> | [R]: "APR=116.73%..... Holy fuck! This is a "Native American owned business operating on a reservation", exempt from US law." |
| 06:46 | < Syk> | from imgur |
| 06:46 | < Syk> | that's why they can charge an APR of 100%+ :P |
| 06:51 | < [R]> | k... |
| 06:51 | < [R]> | Still don't get why I'm being highlighted with all this info. |
| 06:51 | < [R]> | *shrugs* |
| 06:52 | < [R]> | I mean, all I did was post something that had a logic error on it. Namely "42 milion < 5 grand" |
| 07:11 | | himi [fow035@D741F1.243F35.CADC30.81D435] has quit [Operation timed out] |
| 07:17 | | Xon [Xon@9C034E.A7474E.446F1A.DEA144] has joined #code |
| 07:19 | <~Vornicus> | I don't see a logic error there |
| 07:20 | <~Vornicus> | I see a typo. |
| 07:26 | < [R]> | Oh I'm tired |
| 07:27 | < [R]> | I read it as "loan up to $5000" with them paying you that red-highlighted stuff. |
| 07:29 | < Syk> | [R]: ...lol |
| 07:29 | < Syk> | no, I don't think anyone will be giving you a 116% apr :P |
| 07:29 | < [R]> | Nope |
| 07:30 | < Syk> | i'm happy with my bank's 4.65% pa >v> |
| 07:30 | < Syk> | but yeah |
| 07:33 | < [R]> | So massively off-topic, but I might be playing a character in a TTRPG that is not all there, and frequently talks about her dreams that always end up with a very bizzare death. So far I have "head-holes spew flaming rainbows as their face melts off", "pre-emptively riding into the sunset resulting in the sun crushing them between itself and the horizon", "stangled to death by one's own reflection", "fall into the sky and splatter on a very solid cloud". Anyone able |
| 07:33 | < [R]> | to think of other additions? |
| 07:34 | < [R]> | Oh also: "being drunk by a cup of tea" |
| 07:49 | < sshine> | in what university courses are fourier transforms usually taught? |
| 07:49 | < sshine> | I only see fourier series in mathematical analysis. |
| 07:50 | < sshine> | I think there's a course called "signal and image processing" where they're used. but I don't know if they're introduced there. |
| 07:50 | < sshine> | I'm trying to find an introductory text, since this course uses them but forgets to introduce them. |
| 07:56 | <~Vornicus> | linalg or diffeq |
| 08:21 | <~Vornicus> | Or, I suppose, you could ask me |
| 08:23 | <@Alek> | TTRPG? |
| 08:33 | < sshine> | Vornicus, cool. :) right now, my assignment text feels like the manuscript to a star trek episode. |
| 08:37 | <~Vornicus> | it does that. |
| 08:51 | | * Vornicus recommends #math |
| 08:52 | | himi [fow035@0C0840.B22E58.E3471A.E028A1] has joined #code |
| 08:52 | | mode/#code [+o himi] by ChanServ |
| 08:52 | | You're now known as TheWatcher |
| 09:17 | | VirusHome [VirusJTG@Nightstar-09c31e7a.sta.comporium.net] has joined #code |
| 09:19 | | Pandemic [VirusJTG@Nightstar-09c31e7a.sta.comporium.net] has quit [Ping timeout: 121 seconds] |
| 09:24 | | celticminstrel [celticminst@Nightstar-e83b3651.cable.rogers.com] has quit [[NS] Quit: And lo! The computer falls into a deep sleep, to awake again some other day!] |
| 09:47 | < [R]> | Alek: Table-Top RPG (IE a roleplaying game) |
| 09:48 | | thalass [thalass@C2A270.1179B7.313116.A6A5E2] has joined #code |
| 10:23 | <@Alek> | ah, I'm not familiar with the shortening TT. |
| 10:23 | <@Alek> | more with P&P |
| 10:33 | < [R]> | Ah |
| 10:39 | | * thalass idly wonders how he could reliably log in to his home network from the wider internets, without a fixed IP address |
| 10:40 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has joined #code |
| 10:40 | <@froztbyte> | I don't like it much, but teamviewer does that well |
| 10:41 | <~Vornicus> | without a fixed IP address you can get like no-ip to point at your ip and have your router or computer automatically update that |
| 10:43 | <@froztbyte> | there's that too |
| 10:43 | < thalass> | I'm digging through linuxjournal at the moment, i'm sure there was a hack mentioned that would do something but i can't remember |
| 10:44 | <@froztbyte> | issues with it, and why I dislike it: line drops usually introduce a couple of minutes latency on the update, you may need to have a different router, dyndns (as a company) fucking sucks so you definitely want to check the others |
| 10:44 | < [R]> | dyndns? |
| 10:44 | <@froztbyte> | but that's about it |
| 10:44 | <@froztbyte> | if you have a box that's always on the internet, vpn tiems! |
| 10:45 | | * froztbyte is currently just using IPv6 /everywhere/ since that's a helluva lot easier |
| 10:45 | | * froztbyte is going to need to find a new allocation soon though |
| 10:47 | < thalass> | hrm |
| 10:48 | < thalass> | I've been fiddling with Tasker and i think i can use it, a Raspberry Pi, and a bit of bodging with the motor controller, to make the garage door open automatically when i get close to home. :P |
| 10:50 | <@froztbyte> | certainly |
| 10:50 | <@froztbyte> | you could of course also kludge it |
| 10:51 | <@froztbyte> | have tasker upload a file to s3/dropbox/whatever |
| 10:51 | <@froztbyte> | garage unit checks that URL once a minute or so |
| 10:51 | <@froztbyte> | if there's an open command, it opens |
| 10:52 | < thalass> | That could work |
| 10:52 | <@froztbyte> | http://xkcd.com/1209/ AHAHAHAHA |
| 10:55 | < thalass> | *snerk* |
| 11:01 | < sshine> | I like http://xkcd.com/1207/ :) I think there's a business for air drones at some point! |
| 11:01 | < sshine> | I think navigating the air is probably much easier in the long run than building bots that can walk or drive. |
| 11:02 | < sshine> | I mean, a non-military business |
| 11:04 | < thalass> | heh |
| 11:04 | < thalass> | I would think a uav-based courier business could be viable. That, and taxis. Robot taxis that don't drive past you because your fare isn't high enough. >.M |
| 11:04 | < thalass> | er. >.< |
| 11:11 | | * Alek found the MMO that was THE inspiration for the book Omnitopia. o_o |
| 11:21 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has quit [Client closed the connection] |
| 11:33 | | * thalass eyes openrocket, mwahahahas |
| 11:43 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has joined #code |
| 11:46 | < thalass> | aha! found the article. |
| 11:52 | < RichyB> | Dear everybody who ever touched the CSS specification: what the shitting fuck? |
| 11:52 | < RichyB> | Love, Richard. |
| 11:53 | <@froztbyte> | haha |
| 11:57 | <@TheWatcher> | RichyB: Your ideas are intriguing to me and I wish to subscribe to your newsletter. |
| 12:25 | < RichyB> | Okay, so it turns out that my mystery here is that despite the similar names, the behaviours of horizontal margins and vertical margins are utterly different. |
| 12:42 | | thalass [thalass@C2A270.1179B7.313116.A6A5E2] has quit [Ping timeout: 121 seconds] |
| 12:59 | <@TheWatcher> | You're not expecting consistency and sanity in css, are you? |
| 13:01 | < Xon> | RichyB, CSS is cluster fuck which is only going to get better when we have a single browser to rule them all that doesn't give a fuck about that "standard" (10 years on, CSS 2.x still hasn't hit standard status) |
| 13:02 | < RichyB> | "(10 years on, CSS 2.x still hasn't hit standard status)" is not much of an indictment. |
| 13:02 | < RichyB> | Most of the IETF standards that we rely on on a day to day basis have not reached STANDARD status either. |
| 13:03 | < RichyB> | Almost all of the RFCs that you use in practice are still at RECOMMENDATION status. Many are at DRAFT. ? |
| 13:03 | < Xon> | sorry, 15 years not 10 |
| 13:04 | < RichyB> | e.g. the first RFC for IRC is 1459, marked "an Experimental Protocol" |
| 13:04 | < RichyB> | The current RFCs for IRC, 2810 through 2813 inclusive, are marked "Informational". |
| 13:05 | < Xon> | yeah, and they are trying to build css 3.x on a still changing CSS 2.x |
| 13:06 | < Xon> | RichyB, css hasn't even hit RFC status either |
| 13:07 | < RichyB> | Er... yes? Why would that be relevant? |
| 13:08 | < RichyB> | CSS also isn't an ISO, ECMA, ANSI or NIST standard. :P |
| 13:08 | < RichyB> | The RFCs are the IETF's publications and the IETF is primarily interested in wire formats for network communication. |
| 13:09 | < RichyB> | Well, less wire formats, more protocols. |
| 13:09 | < RichyB> | Nobody's going to ever submit CSS to as an RFC to the IETF because it doesn't fit the remit that the IETF is there to fulfil. |
| 13:11 | | * TheWatcher readsup |
| 13:12 | <@TheWatcher> | RichyB: while your point is valid, IRC is probably not the best example, given that anyone that does anything lower-level than "use an IRC client" is quite well aware of how squaamous and rugose it is. |
| 13:12 | <@TheWatcher> | -a |
| 13:13 | | * TheWatcher makes the appropriate cthulhian tentacle wrigglings motions with fingers under his chin |
| 13:14 | < RichyB> | Yes, yes, well okay. |
| 13:14 | < RichyB> | Better examples: RFC 2616, usually referred to as "HTTP/1.1", is a "DRAFT STANDARD". |
| 13:15 | <@froztbyte> | indeed |
| 13:15 | <@TheWatcher> | (I always get tempted to drop the first 'R' in there for some reason) |
| 13:15 | <@froztbyte> | HTTP/1.1 (and I think some HTTP/2 work?) are in heavy flux as far as /real/ things go |
| 13:15 | <@froztbyte> | it's all a bit of a fuckup |
| 13:17 | | Vornicus [vorn@ServerAdministrator.Nightstar.Net] has quit [[NS] Quit: Leaving] |
| 13:21 | | Shiz [mark@42F7BC.C2C1CC.B64DF6.88F7A5] has quit [[NS] Quit: WeeChat 0.4.1-dev] |
| 13:23 | | thalass [thalass@C2A270.1179B7.313116.A6A5E2] has joined #code |
| 13:25 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has quit [Ping timeout: 121 seconds] |
| 13:45 | | * TheWatcher eyes gitlab |
| 13:45 | <@TheWatcher> | ... it amuses me that they're making a github clone, hosting the repository on github. |
| 13:46 | <@TheWatcher> | Also, why, why rails? Ugh. |
| 13:49 | < Syk> | doesnt github use rails |
| 13:50 | <@TheWatcher> | rails and erlang, yes |
| 14:52 | | RichyB [richardb@58734C.5279B7.EA7DF8.107330] has joined #code |
| 15:28 | < sshine> | I wonder if matlab's conv() function secretly does fast fourier transforms. how would I verify this? |
| 15:36 | | ToxicFrog is now known as ToxicFrog|W`rkn |
| 15:40 | | Turaiel[Offline] is now known as Turaiel |
| 15:45 | < JustBob> | I believe that convnfft is the one that does an fft. |
| 15:46 | < JustBob> | http://www.mathworks.com/help/matlab/ref/conv.html?nocookie=true |
| 15:47 | < JustBob> | Looking at that, it doesn't look like an fft to me. |
| 15:49 | < JustBob> | https://ccrma.stanford.edu/~jos/st/Convolution_Theorem.html <- Yeah, digging into it a little more, the conv() function appears to be a direct convolution, not an fft. |
| 15:49 | < JustBob> | Since that particular script displays the difference between the two. :p |
| 15:50 | < JustBob> | (Note, convnfft is apparently a seperate module, not baked into matlab.) |
| 16:20 | | thalass [thalass@C2A270.1179B7.313116.A6A5E2] has quit [Ping timeout: 121 seconds] |
| 16:59 | | Turaiel is now known as Turaiel[Offline] |
| 17:05 | | Turaiel[Offline] is now known as Turaiel |
| 17:20 | < Xires> | anyone know if there's a common C interface for dealing w/ ext2/ext3 filesystem attributes? |
| 17:21 | <@Tamber> | Off the top of my head, I *think* so. I could be wrong, though. |
| 17:21 | <@Tamber> | One moment. |
| 17:22 | <@Tamber> | See getxattr(2), setxattr(2), I think. |
| 17:22 | < Xires> | I'm having quite a bit of trouble tracking anything down |
| 17:23 | < Xires> | Tamber; ty..I wonder why I couldn't find anything via google |
| 17:24 | <@Tamber> | Because sometimes, to think of the right keywords, you have to know what you're trying to find. ~.~ |
| 17:32 | < RichyB> | getxattr and setxattr aren't specific to ext2/ext3/ext4 |
| 17:33 | <@Tamber> | Even if they don't do the job, it might point in the direction of the things that do. |
| 17:34 | < RichyB> | other filesystems like XFS have xattr support too ^_^ |
| 17:42 | < Xires> | sadly, not what I was after..I'm basically looking for a C interface to lsattr/chattr commands |
| 17:42 | < Xires> | looks like one doesn't exist so I'll have to write one myself |
| 17:44 | <@Tamber> | Ah |
| 17:47 | < RichyB> | Ah, not extended attributes. |
| 17:47 | | Turaiel is now known as Turaiel[Offline] |
| 17:48 | < [R]> | Xires: that package should require a special lib (as the git repo for it includes said lib), you can use the function in that. |
| 17:48 | < Xires> | [R]; that's what I was thinking so I've been looking at the e2fsprogs repo |
| 17:49 | < [R]> | http://git.kernel.org/cgit/fs/ext2/e2fsprogs.git/tree/lib/ext2fs/ext_attr.c <-- the functions you want (can't find the header ATM, sorry) |
| 17:49 | < Xires> | fsetflags & fgetflags seems to be what it uses |
| 17:49 | < [R]> | Oh |
| 17:49 | < [R]> | nm me then :p |
| 17:50 | < Xires> | I appreciate the extra google-fu |
| 17:53 | | Syk is now known as syksleep |
| 17:59 | < Xires> | [R]; for future reference, e2p.h is the header that defines the functions and that can be installed w/ the e2fslibs development package |
| 17:59 | | AnnoDomini is now known as Wires |
| 18:00 | < [R]> | Cool, thanks |
| 18:41 | < sshine> | JustBob, thanks for looking into the conv/fft thing! |
| 19:05 | | celticminstrel [celticminst@Nightstar-e83b3651.cable.rogers.com] has joined #code |
| 19:05 | | mode/#code [+o celticminstrel] by ChanServ |
| 19:13 | | Kindamoody|out is now known as Kindamoody |
| 19:14 | < RichyB> | Derakon[AFK]: BTW I had a really strikingly obvious idea the other day - a circular queue buffer is *way* simpler if its implementation just tracks the current head and number of filled-in elements instead of tracking the current head and tail. |
| 19:15 | < RichyB> | Derakon[AFK]: so I rewrote the queue bits of https://github.com/RichardBarrell/heatmap-for-Derakon/blob/master/_cmap.c and now they feel much simpler; the invariants seem much less subtle. ? |
| 19:38 | <@froztbyte> | RichyB: I'm gonna take a guess and say that http://pages.cs.wisc.edu/~vijayc/nofs.htm might appeal to you |
| 19:38 | <@froztbyte> | at least in some fashion |
| 19:39 | <@froztbyte> | (there are some obvious issues in the sense of full surface scans and such, but the basic premise is kinda cool) |
| 19:39 | < RichyB> | wat |
| 19:39 | | * RichyB scratches nogging |
| 19:40 | <@froztbyte> | at which part? :) |
| 19:40 | < RichyB> | froztbyte: yes, that's interesting, I'm going to have to read it now because I can't see how on Earth their consistency-maintaining thing is actually not impossible? |
| 19:40 | <@froztbyte> | a reflink is considered valid only if it points both ways |
| 19:41 | < RichyB> | Okay, so it can stay internally consistent, it just can't offer any guarantees upwards stronger than "mounting the filesystem again will always succeed"? |
| 19:41 | <@froztbyte> | a little bit more than that |
| 19:41 | < RichyB> | But the layers below it are allowed to throw away utterly-arbitrary writes. |
| 19:41 | <@froztbyte> | internally consistent, and that whatever you get told is, is definitely there |
| 19:42 | < RichyB> | No, it can't possibly implement safe working fsync() if the layers under it don't. |
| 19:42 | <@froztbyte> | I would like to see some further work on scaling this up to a couple hundred gigs, though |
| 19:43 | < RichyB> | The best you could possibly do is "if the disk subsystem under me has bothered to flush your write then it'll definitely be present when the system comes back up". |
| 19:43 | <@froztbyte> | one of the ideas I had cursorily thought of was that you could write a "hints" file |
| 19:43 | <@froztbyte> | not a true reference source, but a thing that'll tell you where you can seek to if you wanted to run spot checks |
| 19:44 | < RichyB> | Hints about what? |
| 19:44 | <@froztbyte> | of course, surface scans are only an issue with SRDs, so maybe that's irrelevant |
| 19:44 | <@froztbyte> | RichyB: about where your slabs and reference nodes lie |
| 19:44 | <&ToxicFrog|W`rkn> | RichyB: I don't think that's what fb meant by "whatever you get told is there is actually there" |
| 19:44 | <@froztbyte> | ah, yes |
| 19:44 | <@froztbyte> | on that point |
| 19:44 | <&ToxicFrog|W`rkn> | It's "the metadata will always be consistent with the file contents" - you'll never get directory entries that point to garbage inodes, for example |
| 19:44 | <&ToxicFrog|W`rkn> | (disclaimer: I'm only a few pages in) |
| 19:45 | <@froztbyte> | all of the query mechanisms in the filesystem are built on having checked for consistency |
| 19:45 | < RichyB> | Right, so it's always *internally* consistent. Cool. |
| 19:45 | <@froztbyte> | so `ls` and friends |
| 19:45 | <@froztbyte> | as to what you put onto the disk, that's the part where they say "well, we can't control what might happen here, so we're chucking that entire situation out the window" |
| 19:46 | <@froztbyte> | ZFS "solves" this particular problem with a ZIL |
| 19:46 | <@froztbyte> | (ZFS Intent Log) |
| 19:46 | < RichyB> | Well yes |
| 19:46 | <@froztbyte> | which people tend to put on SSDs |
| 19:46 | <@froztbyte> | other systems solve it with RAID batteries |
| 19:46 | <@froztbyte> | but none of those is infallible |
| 19:46 | < RichyB> | The ZIL is a write-ahead log. They swiped the idea from database design. |
| 19:47 | <@froztbyte> | your SSD can burn out, your raid controller derps, a drive forgets its own firmware, etc |
| 19:48 | <@froztbyte> | I *think* the NOFS people had the idea that the write-validation should be moved into application logic |
| 19:48 | < RichyB> | You use a proper logzilla and you test and monitor the battery on your RAID card. |
| 19:48 | < RichyB> | Nothing's ever going to be perfectly resilient against hardware failure; someone could simply crash an aeroplane into your entire data centre. |
| 19:49 | <@froztbyte> | or comets |
| 19:49 | <@froztbyte> | or orbital bombardment |
| 19:49 | < RichyB> | Indeed. At this point we rely on a thing called "backups". |
| 19:49 | <@froztbyte> | in fact, now I want to know if I know anyone with DCs planned against various sorts of bombardment |
| 19:49 | < RichyB> | er |
| 19:49 | < RichyB> | "Disaster recovery" will do for disasters. |
| 19:49 | < JBeshir> | froztbyte: Lots of people have. |
| 19:49 | < JBeshir> | It's called "have multiple DCs" |
| 19:49 | < RichyB> | Backups only become the *only* option when the data corruption comes from "typed rm -fr on the wrong directory". |
| 19:50 | <@froztbyte> | RichyB: backups/DR/etc, yeah |
| 19:50 | < JBeshir> | Seriously, that's part of the point of really geographically separated fault tolerant systems |
| 19:50 | <@froztbyte> | JBeshir: no, not that |
| 19:50 | <@froztbyte> | I'm talking about "DCs that can hold off people with tanks" |
| 19:50 | < RichyB> | You mean DCs that are specifically built to survive nuclear strikes? Yes. |
| 19:50 | < JBeshir> | Why would you do that when you can have multiple DCs? |
| 19:50 | | * froztbyte knows some people who might have such datacenters |
| 19:50 | <@froztbyte> | JBeshir: to make them even harder to kill |
| 19:50 | < JBeshir> | Why do that when you can have multiple DCs? |
| 19:51 | < JBeshir> | It's been done, but it's a gimmick |
| 19:51 | < RichyB> | There's a bunch; some are military, others are just built by people who happened to buy up old military bases. |
| 19:51 | <@froztbyte> | if all your DCs are easy to kill, I'll just target them all |
| 19:51 | < RichyB> | There's a famous data centre somewhere in mainland Europe that's built into a missile silo. |
| 19:51 | < JBeshir> | That doesn't seem related to the "comets" example |
| 19:51 | < RichyB> | Generally you do that because it's cool moreso than because it's useful. |
| 19:51 | <@froztbyte> | yeah, cyberbunker etc |
| 19:51 | < JBeshir> | I think very few people plan for defence for military attack |
| 19:52 | <@froztbyte> | JBeshir: comets were just a fun example |
| 19:52 | <@froztbyte> | but very few people have the need to design for such extreme circumstances |
| 19:52 | <@froztbyte> | (yet) |
| 19:52 | < JBeshir> | They're the kind of example people actually care about- earthquakes, fires, hurricanes, etc. |
| 19:52 | < RichyB> | Yeah, it's just that there are a few places that are capable of surviving artillery because the owners built them into old military installations which they bought because they're really cool. :) |
| 19:52 | < RichyB> | OTOH |
| 19:53 | < JBeshir> | And they deal with that with multiple DCs. No one deals with coordinated malicious military attacks on all their DCs except probably the military |
| 19:53 | < RichyB> | multiple data-centre survivability is a very real need |
| 19:53 | <@TheWatcher> | froztbyte: And of course, none of them are prepared for CASE NIGHTMARE GREEN, so... |
| 19:53 | <@froztbyte> | JBeshir: I know two companies who do :) |
| 19:53 | < RichyB> | All along the south and east coast of Australia, emergency services systems have to contend with yearly tsunamis taking our their DCs. |
| 19:54 | < JBeshir> | froztbyte: Good for you? That sounds like a massive waste of money unlikely to be actually effective, though |
| 19:54 | < JBeshir> | I mean, cyberbunker held off their local council for like, an hour |
| 19:54 | <@froztbyte> | TheWatcher: that's just because those damned lazy physicists haven't invented portal guns, found the right mountains, found us other planets, etc |
| 19:54 | < JBeshir> | Actual military action against them would probably just be funny |
| 19:54 | <@froztbyte> | JBeshir: sure, I don't dispute that it's a silly amount of resources to expend |
| 19:55 | <@froztbyte> | but for them they've obviously somehow considered it worth it |
| 19:55 | <@froztbyte> | my interest in it is the technical challenges behind such things |
| 19:55 | < JBeshir> | Well, the technical challenge is basically that no one really does it aside the military and everyone else is just posing with pictures of a cool bunker. |
| 19:56 | < JBeshir> | If you want to pay extra for those pictures you can, but you should probably not actually believe that cyberbunker will protect you against a hostile military force. |
| 19:56 | < RichyB> | I think that the technical challenges are kind of boring. Dig a hole under some granite, lay lots of fibre-optic cables, have your own power supply. |
| 19:56 | < JBeshir> | Don't have family members held hostage to get you to open the door... |
| 19:56 | < RichyB> | Yep. |
| 19:57 | < RichyB> | The multi-DC solution is the one that's actually both realistic and technologically interesting. |
| 19:57 | < RichyB> | Technologically interesting because it puts you into the world of distributed systems (yay!) |
| 19:57 | <@froztbyte> | mmm |
| 19:57 | <@froztbyte> | building larger span networks is a bit meh, though |
| 19:57 | <@froztbyte> | I mean |
| 19:58 | < RichyB> | realistic because it's actually pretty cheap to build more small data centres than anyone can be bothered to find and blow up. |
| 19:58 | <@froztbyte> | routing and such isn't really a pain in the ass |
| 19:58 | <@froztbyte> | if you want to do large L2 spans, I'll ask you why, then hit you in the face with a shovel if you don't answer "I don't actually know" |
| 19:58 | < RichyB> | Also realistic because (moderate numbers of medium-sized DCs) it's the approach that the tsunami-proof emergency services applications are taking. :) |
| 19:59 | < JBeshir> | I don't think distributed systems generally use large L2 spans |
| 19:59 | <@froztbyte> | JBeshir: man, this "cloud" shit.... |
| 19:59 | <@froztbyte> | so, for a little while, "fabric" was a nice big buzzword |
| 20:00 | <@froztbyte> | which meant "larger-tha-you-should-actually-have L2 span", with a couple thousand storage nodes interacting on this |
| 20:00 | < JBeshir> | Weird |
| 20:00 | <@froztbyte> | ("fabric" because it's what stitches it all together, geddit?!) |
| 20:00 | <@froztbyte> | people seem to have finally realized this is a dumb idea |
| 20:00 | <@froztbyte> | but that's how a lot of the vmware shops ran |
| 20:01 | <@froztbyte> | vmware + nexus + fabric + a couple of EMCs or whatever |
| 20:01 | < JBeshir> | It's notable that this kind of fault-tolerant, lots of nodes thing was the original idea of the Internet |
| 20:01 | < RichyB> | ^ e.g. distributed hash tables like Chord and family pretty much live entirely at l7, no weird mucking around with ethernet. |
| 20:01 | < JBeshir> | Rather than try to have communicating places reinforced to be invulnerable, just route around failures |
| 20:01 | <@froztbyte> | I mean, it's understandable how it happened |
| 20:01 | <@froztbyte> | handing VMs off with iscsi and doing block-level replication in a VLAN does make things easier |
| 20:02 | <@froztbyte> | even intra-/inter-area transfer becomes a lot more manageable with that |
| 20:02 | <@froztbyte> | due to arp takeovers and such |
| 20:02 | < RichyB> | froztbyte: bah, I wish I had enough money to run all of our applications that way. |
| 20:02 | <@froztbyte> | but man, the issues it brings with... |
| 20:02 | <@froztbyte> | RichyB: it's not really worth it |
| 20:02 | < RichyB> | no, no, no |
| 20:02 | <@froztbyte> | or am I misunderstanding you? |
| 20:02 | < RichyB> | I wouldn't *do it*, I'd just like to have that much money. ;0 |
| 20:02 | < RichyB> | ;) |
| 20:02 | <@froztbyte> | oh |
| 20:03 | <@froztbyte> | yeah |
| 20:03 | <@froztbyte> | well, go work for a corporate ;p |
| 20:03 | < RichyB> | Same principle as "I wish I had enough money to run Oracle instead of Postgres." |
| 20:03 | < RichyB> | ;) |
| 20:03 | <@froztbyte> | hehe |
| 20:04 | <@froztbyte> | anyway, to get back to my original starting point |
| 20:04 | <@froztbyte> | it's cool to see that people are taking somewhat alternative approaches in filesystem design |
| 20:04 | < RichyB> | Yeah, what were we talking about? |
| 20:04 | < RichyB> | Mmyeahokay. |
| 20:04 | < RichyB> | Please don't discount ZFS in that though. |
| 20:04 | <@froztbyte> | oh, no, certainly not discounting it |
| 20:04 | < RichyB> | IMHO ZFS is very innovative too. |
| 20:04 | | Kindamoody is now known as Kindamoody[zZz] |
| 20:05 | <@froztbyte> | in fact, I recently became quite familiar with the internal zfs datastructures |
| 20:05 | <@froztbyte> | (that which is known as zdb) |
| 20:05 | < RichyB> | The key thing in ZFS is that they changed the interface to storage. Most file systems present a POSIX interface and consume a raw-blocks interface. |
| 20:05 | | * froztbyte has recently been getting a lot more involved in storage innards for no other reason than the fact that he's completely fucked in the head |
| 20:05 | <@froztbyte> | RichyB: ye |
| 20:06 | < RichyB> | ZFS instead presents a POSIX interface and consumes a transactions-over-objects interface (which in turn consumes a raw-blocks interface) |
| 20:06 | <@froztbyte> | other people are still barely catching up |
| 20:06 | <@froztbyte> | and I'm not talking just filesystems |
| 20:06 | < RichyB> | which in retrospect is such an obviously good idea that I wish I had a fucking time machine. :) |
| 20:06 | <@froztbyte> | the work the guys did at Sun is *hawt* |
| 20:06 | <@froztbyte> | have you seen SmartOS? |
| 20:06 | < RichyB> | Never tried using it. |
| 20:06 | <@froztbyte> | http://smartos.org/ |
| 20:07 | < RichyB> | I've seen some of Ben Rockwood (cuddletech guy's) presentations and tech talks on it though |
| 20:07 | <@froztbyte> | roughly stated, it's where the Sun brains ended up |
| 20:07 | < RichyB> | ...which leads in to one of the funniest things I've ever seen in a tech talk, actually. |
| 20:07 | <@froztbyte> | Sun, and Yahoo |
| 20:07 | <@froztbyte> | Real too, actually |
| 20:07 | < RichyB> | BenR's standing at the front of the room demonstrating the virtualisation system (port of KVM, I think?) that they have built into SunOS |
| 20:08 | <@froztbyte> | tech companies who did some really fucking cool things that not many people know about |
| 20:08 | < RichyB> | er, SunOS? SmartOS. Daughter of SunOS. |
| 20:08 | <@froztbyte> | hmm |
| 20:08 | <@froztbyte> | that might be Illumos? |
| 20:08 | <@froztbyte> | possibly SmartOS, depending on timeline |
| 20:08 | <@froztbyte> | but it sounds like SmartOS, because I think you're referring to Zones |
| 20:08 | < RichyB> | anyway, he's going through some of the OS features, and it happens that you can see the output of "ifconfig" on his terminal for a few minutes while he's talking |
| 20:09 | < RichyB> | and half the room, specifically the half that consists of old beardy Solaris types (as opposed to young eager kids) starts giving him shit |
| 20:09 | <@froztbyte> | rofl |
| 20:10 | < RichyB> | and taking the piss out of him for using the Intel e1000 driver. |
| 20:10 | <@froztbyte> | I'm .... weird |
| 20:10 | | * froztbyte technically falls into the latter group, but often feels far more affinity to the former |
| 20:10 | < JBeshir> | Haha |
| 20:11 | < RichyB> | (background: that kinda makes sense, because its means that he's left the KVM network device as the default emulating-a-piece-of-hardware driver instead of setting up the faster paravirtualised network interface) |
| 20:11 | < RichyB> | (still... it turns into a small flamewar ^_^) |
| 20:11 | <@froztbyte> | on that note |
| 20:12 | <@froztbyte> | I think I know why that happened |
| 20:12 | <@froztbyte> | last time I tried SmartOS I couldn't get it to actually boot, because it didn't have drivers for any but 3 NICs... |
| 20:12 | < RichyB> | froztbyte: eh makes sense. Old people did a lot of shit that the world has forgotten a lot of. ? |
| 20:12 | <@froztbyte> | well, I did later get it to boot, but in a much degraded fashion |
| 20:13 | <@froztbyte> | RichyB: aye |
| 20:13 | <@froztbyte> | but it also leads to me having a lot of disdain for things |
| 20:13 | <@froztbyte> | this is where my dislike of shit like node.js comes most strongly from |
| 20:13 | <@froztbyte> | people (paraphrased for effect): omfg this node.js thing is awesome! you can like do async and shit!!~ |
| 20:14 | <@froztbyte> | me: fisher-price, my first event machine. *snooze* |
| 20:14 | | * froztbyte has a (perhaps irrational) dislike for times when people intentionally don't have breadth of knowledge |
| 20:16 | < RichyB> | I can see the argument for nodejs. If you want to run websockets stuff then it tends to work nicely because you're just appending closures to callback lists rather than burning up threads. |
| 20:17 | < RichyB> | The performance looks amazing if you're coming from something that sucks fat rocks through a straw with a GIL, like Python or (non-JRuby) Ruby. |
| 20:17 | <@froztbyte> | remember the old advice about when people ask how they learn socket programming? |
| 20:17 | <@froztbyte> | I feel that the same likely applies to websockets |
| 20:17 | < JBeshir> | Goddamn it |
| 20:17 | < RichyB> | OTOH, I'm used to GHC Haskell, where you can just use threads and the fuckin' API is actually threads dammit. |
| 20:18 | <&ToxicFrog|W`rkn> | froztbyte: what is the old advice? |
| 20:18 | < JBeshir> | I work on a Paxos implementation |
| 20:18 | <@froztbyte> | we've just gotten enough resources to make it hurt less |
| 20:18 | <@froztbyte> | ToxicFrog|W`rkn: "you probably don't" |
| 20:18 | < RichyB> | But in GHC's run-time, there's an m:n userland threading implementation built in. |
| 20:18 | < JBeshir> | And then I find this https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf draft paper of a simpler version of Paxos published one month ago, long after I started work |
| 20:18 | <@froztbyte> | ToxicFrog|W`rkn: which is to mean "you likely want to know how to connect two or more networked systems together. that's what tcp/ip is for" |
| 20:18 | <&ToxicFrog|W`rkn> | RichyB: my objection to async is that you can also use coroutines to implement something that looks synchronous but gives you async performance and doesn't spawn a thread per request and get you the best of both worlds |
| 20:18 | < RichyB> | You can spawn millions of threads in GHC Haskell pretty cheaply; it's just appending closures to the ends of run queues! |
| 20:19 | <&ToxicFrog|W`rkn> | I.e. you get good performance and your code isn't a total nightmare to follow |
| 20:19 | < JBeshir> | What ToxicFrog|W`rkn said |
| 20:19 | < JBeshir> | If your language doesn't support it, use a superior language |
| 20:19 | <&ToxicFrog|W`rkn> | Unless you're using something like Thoth or CacheOS in which case you just spawn a few hundred thousand threads and rock out |
| 20:19 | < RichyB> | ToxicFrog|W`rkn: my objection to coroutines is that you also use an m:n threading model, have an actually-synchronous-thank-you-very-much programming model and have the same low overhead. |
| 20:19 | < JBeshir> | No, no |
| 20:20 | < JBeshir> | You want actually-asynchronous |
| 20:20 | < JBeshir> | Which looks like synchronous |
| 20:20 | < JBeshir> | That's the wrong way around |
| 20:20 | < RichyB> | I am deeply unimpressed by the tangliness of things like Tornado having used GHC. |
| 20:20 | <@froztbyte> | I don't know why you'd ever want something that looks like the one and is the other |
| 20:20 | <&ToxicFrog|W`rkn> | RichyB: my point here is that you can have a synchronous programming model that uses a high-performance async implementation under the hood, if you are working in a domain where async gives you performance benefits. |
| 20:20 | < RichyB> | froztbyte: you want something that looks like the convenient one but performs as well as the inconvenient one does. |
| 20:20 | < JBeshir> | froztbyte: That's a problem with your knowledge, bluntly |
| 20:21 | <&ToxicFrog|W`rkn> | froztbyte: what RichyB said |
| 20:21 | <@froztbyte> | JBeshir: well, my first impression is exactly because of that |
| 20:21 | < RichyB> | froztbyte: the former makes it cheaper to program; time is money. The latter makes it run quicker; CPU time is electricity. |
| 20:21 | <@froztbyte> | if I were to use a thing that appeared async, but was actually sync |
| 20:21 | <@froztbyte> | and I wasn't aware of that |
| 20:21 | <@froztbyte> | then I'd be terribly fucking confused |
| 20:21 | < JBeshir> | Nothing would happen |
| 20:21 | < RichyB> | ToxicFrog|W`rkn: yyyeah, GHC's threading (also Go's, they either copied or hit on the same idea) gets you exactly that. |
| 20:21 | < JBeshir> | No, you wouldn't, really |
| 20:21 | <@froztbyte> | of course I would |
| 20:22 | < JBeshir> | No, you would not |
| 20:22 | <&ToxicFrog|W`rkn> | RichyB: it's not exactly a new concept; I believe Erlang threads behave similarly, and Scala Actors definitely do |
| 20:22 | < RichyB> | But you don't have any semantic complexity introduced by it; GHC's green threads are just ordinary IO actions. |
| 20:22 | <@froztbyte> | I'd run a tutorial, see "okay, I think I see how this works" |
| 20:22 | < JBeshir> | These systems provide the semantics of sync with the performance of async because they implement those semantics using async behind the scenes |
| 20:22 | <@froztbyte> | then I'd write some of my own code that looks kinda like it |
| 20:22 | <@froztbyte> | but doesn't work quite right |
| 20:22 | < JBeshir> | It should work |
| 20:22 | <@froztbyte> | and I'd be terribly lost as to why |
| 20:22 | <@froztbyte> | JBeshir: /should/ does not imply /will/ |
| 20:22 | <@froztbyte> | JBeshir: the case I'm making here is for the garden variety developer |
| 20:22 | <&ToxicFrog|W`rkn> | froztbyte: you're confused. It doesn't "look like sync but is actually async" in the sense you seem be thinking, which is "it looks like a sync API but behaves like an async one" |
| 20:22 | < JBeshir> | froztbyte: The case you're making is ignorant |
| 20:22 | <&ToxicFrog|W`rkn> | It behaves in every respect like a sync API. |
| 20:22 | < JBeshir> | froztbyte: Real systems of this nature exist |
| 20:23 | <&ToxicFrog|W`rkn> | It looks like a sync API. |
| 20:23 | < JBeshir> | They work |
| 20:23 | <@froztbyte> | JBeshir: which is what they are |
| 20:23 | | Tamber [tamber@furryhelix.co.uk] has left #code ["The sun is the same in a relative way, but you're older; And shorter of breath and one day closer to death..."] |
| 20:23 | <@froztbyte> | they *are* ignorant |
| 20:23 | <@froztbyte> | that's why I'm using this example |
| 20:23 | <&ToxicFrog|W`rkn> | But under the hood, it is using fewer actual threads than you have written and is managing them using high-performance async APIs. |
| 20:23 | < JBeshir> | No. |
| 20:23 | < RichyB> | ToxicFrog|W`rkn: to be clear, I'm objecting to the idea that the fact that it's backed by coroutines would be exposed to business logic. GHC, Go, Erlang and so on all manage good evented green thread implementations that don't expose their green-ness. ? |
| 20:23 | <@froztbyte> | JBeshir: let me step back |
| 20:23 | < JBeshir> | Their lack of knowledge does not cause problems of this nature. It is your lack of knowledge making you think they would. The semantics provided by these systems as they actually do exist match the sync semantics you would expect. |
| 20:23 | <@froztbyte> | JBeshir: I'm talking about the case where someone's coming from writing PHP, their PHP is falling over, and they saw a blogpost about this great LangX thing that's supposed to roflscale |
| 20:23 | < JBeshir> | They just do so more efficiently. |
| 20:24 | <&ToxicFrog|W`rkn> | RichyB: yeah, I think we're in agreement here. My argument was "if your language is worth using, you can get stuff that looks like threads but is actually using coroutines or greenthreads under the hood, and thus get the performance benefits without the spaghetti" |
| 20:24 | <@froztbyte> | or their boss has seen the post, and told them they need to rewrite the application in LangX ritenao |
| 20:24 | < RichyB> | ToxicFrog|W`rkn: I'll drink to that. |
| 20:24 | < JBeshir> | The worst you would have to worry about is your 'threads' actually being cooperatively scheduled |
| 20:24 | <@froztbyte> | (me, personally, would just go and learn more about the things in question, because I like learning) |
| 20:25 | < RichyB> | Ideally, your language multiplexes those over multiple OS threads; but that's not mandatory just yet because I'm still pretty happy to spin up one framework instance per CPU core and put haproxy in front of all of them. |
| 20:25 | <&ToxicFrog|W`rkn> | froztbyte: you are completely missing the point here, which is that the API behaves like a one-thread-per-request synchronous setup. If it doesn't the implementor has fucked up large. |
| 20:26 | < RichyB> | On the other other other other other hand, if you have really cheap OS threads, fuckit, just use one-thread-per-request. |
| 20:26 | <&ToxicFrog|W`rkn> | The fact that, under the hood, it is multiplexing coroutines or greenthreads onto a thread pool is an implementation detail that gets you improved performance. |
| 20:26 | <@froztbyte> | ToxicFrog|W`rkn: people often write bugs instead of software, which is where the confusion of the end user is likely to happen |
| 20:26 | < RichyB> | Varnishd does that and it runs like greased lightning, though it tends not to like long-open requests quite as much as nginx does. |
| 20:26 | < JBeshir> | Aside that your coroutines/green threads may be cooperatively scheduled. |
| 20:26 | < RichyB> | pthreads are cheaper than you'd expect, especially on modern Unixes. ^_^ |
| 20:26 | <&ToxicFrog|W`rkn> | froztbyte: what are these bugs that would appear in this instance but not in a true 1:1 threading setup? |
| 20:27 | <@froztbyte> | race conditions leading to mixup, rather than the system just falling over itself in locking? |
| 20:28 | < JBeshir> | That doesn't really make sense. |
| 20:28 | < JBeshir> | Race conditions can lead to basically the same kind of things with both real OS threads and 'green' threads. |
| 20:28 | <@froztbyte> | mmm |
| 20:28 | <&ToxicFrog|W`rkn> | It really doesn't. |
| 20:29 | <@froztbyte> | okay well, I guess that for now I'll have to accede the point until I can better explain what I have in mind |
| 20:30 | <@froztbyte> | as I said, I'm not trying to argue specifically, there were just some cases that I had in mind where it seems like things could lead to mass confusion |
| 20:31 | <@froztbyte> | (another factor might also be that I've come to know people in the telco space that, at the end of it, are just exceptionally bad programmers) |
| 20:32 | <&ToxicFrog|W`rkn> | Yeah, but those exceptionally bad programmers would be making the same mistakes in a true sync environment. |
| 20:32 | <&ToxicFrog|W`rkn> | I would be interested if you could explain some of the cases you had in mind, though. |
| 20:32 | < RichyB> | Multiple independent threads of control are already scary enough regardless of whether they're synchronous or not. :) |
| 20:33 | < JBeshir> | Synchronous is easier. |
| 20:33 | < JBeshir> | But yeah. |
| 20:33 | <@froztbyte> | ToxicFrog|W`rkn: I don't think I have correct language for communicating what's in my head |
| 20:33 | < RichyB> | OTOH it helps if you're communicating exclusively through a database that's set to enforce some approximation of sequential consistency. |
| 20:33 | <@froztbyte> | some reading over time should rectify this |
| 20:34 | < JBeshir> | What we're discussing is basically an optimised version of a sync environment which combines making threads very cheap so long as they're not making system calls, and implements its standard library's synchronous network I/O, behind the scenes, by having a single thread in the background make async system calls and wake up the lightweight threads when their I/O is done. |
| 20:35 | < JBeshir> | It's semantically equivalent to working in a one-thread-per-connection synchronous I/O model, but not prohibitively expensive. |
| 20:35 | < JBeshir> | (It's roughly as performant as asynchronous I/O) |
| 20:35 | <@froztbyte> | but also, as a result of this |
| 20:36 | <@froztbyte> | I'm now looking at my next couple of weeks again, pondering which days I could reassign |
| 20:36 | | * froztbyte still wants to play with writing some basic crappy OSs just for having done so |
| 20:36 | <&ToxicFrog|W`rkn> | RichyB: they become a lot less scary when you're using message passing, IMO |
| 20:36 | < RichyB> | JBeshir: the implementation doesn't have to quite be that. You don't need to move all I/O onto a single thread, nor implement I/O by pushing messages. |
| 20:36 | <@froztbyte> | (and, longterm, I still want to build a pressure-based water-driven CPU sometime) |
| 20:36 | <&ToxicFrog|W`rkn> | pthreads and similar lock-modify-unlock, shared-memory APIs have scared a lot of people away from concurrency :( |
| 20:36 | < JBeshir> | RichyB: Well, no. All the behind the scenes parts can be varied quite a lot. |
| 20:37 | <@froztbyte> | (yes, I know that's batshit crazy hard) |
| 20:37 | <@froztbyte> | (perhaps even not doable) |
| 20:37 | < RichyB> | JBeshir: you can actually do your I/O in any of the threads that're running user code; you implement a blocking "read()" call by setting the file descriptor nonblocking, then if the read() returns EWOULDBLOCK, you descheduling the user thread and add that file descriptor to the working set for an epoll() or kqueue() queue. |
| 20:38 | < RichyB> | er |
| 20:38 | < RichyB> | *you deschedule the green thread |
| 20:38 | < RichyB> | froztbyte: sluices that open and close each other? Sounds entirely doable. Just don't try to make the instruction set too big. ;) |
| 20:38 | < JBeshir> | RichyB: I think that's something the implementation I was thinking of (an earlier version of the Go implementation before it got smarter) actually does, yeah. |
| 20:39 | <@froztbyte> | RichyB: I want to build it at reasonable-bit precision, with a couple of registers, and large enough to be hilarious |
| 20:39 | <@froztbyte> | RichyB: which means the effects of gravity come into play as well |
| 20:39 | <@froztbyte> | the very idea originated in a joke |
| 20:40 | < RichyB> | froztbyte: well, less of a joke and more of Neal Stephenson wrote about it in The Diamond Age. :D |
| 20:40 | | * froztbyte has not read that |
| 20:40 | | * froztbyte has, in fact, still not read any Neal Stephenson |
| 20:40 | <&ToxicFrog|W`rkn> | RichyB: that's pretty much what copas (the sync-on-async library for lua) does as well. It's not very sophisticated but it's dead easy to implement. |
| 20:40 | < RichyB> | In that novel, one of the characters gets taught computer science from gates on upwards by playing through a big fantastical RPG. |
| 20:40 | <@Alek> | frozt: DO IT! |
| 20:41 | <@froztbyte> | Alek: please quickly go read about VHDL |
| 20:41 | < RichyB> | The videogame she's playing demonstrates logic gates by water sluices. |
| 20:41 | <@froztbyte> | Alek: then you'll understand why I haven't even gotten close to this yet |
| 20:41 | <@froztbyte> | RichyB: interesting |
| 20:41 | <@froztbyte> | RichyB: I may then read this just because |
| 20:41 | <@froztbyte> | although I'll likely end up reading it because reputable people have told me to read a bunch of NS |
| 20:41 | < RichyB> | ToxicFrog|W`rkn: I'm really not seeing what's "unsophisticated" about that mechanism. I'm not aware of any alternative that goes faster. :) |
| 20:42 | < RichyB> | The only *good* reason to not read Neal Stephenson is if you haven't read Charles Stross yet. ? |
| 20:42 | < RichyB> | IMHO Stephenson is excellent, Stross is better. |
| 20:42 | <@froztbyte> | yeah haven't done that either ;p |
| 20:42 | < RichyB> | (Maybe controversial.) |
| 20:42 | <&ToxicFrog|W`rkn> | RichyB: so you should read Stross -second- |
| 20:43 | < RichyB> | ToxicFrog|W`rkn: oh, working up to it? Yeah okay. |
| 20:43 | < RichyB> | TBH they write different kinds of fiction. |
| 20:45 | <@Alek> | no, I meant GO READ STEPHENSON |
| 20:46 | <@Alek> | Stross also. |
| 20:46 | < RichyB> | Is Doctorow on #code's approved-writers list? |
| 20:46 | <@Alek> | probably. |
| 20:46 | | * froztbyte has read some Doctorow and enjoyed it |
| 20:46 | < RichyB> | I found his own writing pretty good, his collaborations excellent. |
| 20:47 | <@froztbyte> | his writing is very good meet-in-the-middle work |
| 20:47 | < RichyB> | I think he does better on collaborations than on his own. |
| 20:47 | <@froztbyte> | Little Brother, for instance, was a pretty damn good intro to various concepts for Joe Average |
| 20:47 | <&ToxicFrog|W`rkn> | RichyB: the main thing that comes to mind is that there are various more sophisticated things you can add on top of that, like dynamically allocating more true threads up to the number of threads doing CPU-heavy computation |
| 20:47 | <@froztbyte> | (at least, I consider it to be) |
| 20:49 | < RichyB> | Why bother doing it dynamically? Just start as many kernel threads as you have SMP cores and leave the excess threads sitting blocked waiting for some user threads to get spawned. |
| 20:50 | < RichyB> | If they don't get an opportunity to do anything, you're looking at about 8kB/core plus a few hundred microseconds. |
| 20:55 | < JBeshir> | One answer is that communication and locking can be implemented much faster if the coroutine/lightweight threads don't actually run simultaneously. |
| 20:56 | < JBeshir> | For this reason programs which aren't compute heavy but use lightweight threads for structuring code can sometimes run faster if left to a single core. |
| 21:11 | | Wires is now known as AnnoDomini |
| 21:18 | < RichyB> | That's a fairly neat idea, yes. Track your own load average and then, when it climbs above 1.0 for a while, switch from elided-locks to actual-locks. |
| 21:18 | < RichyB> | *and turn the extra threads on. |
| 21:19 | < RichyB> | You could do it really nicely with dynamic code modification. |
| 21:20 | < RichyB> | Use proper compare-and-swap instructions for everything, even in single-threaded mode, but every CMPXCHG instruction has a small NOP sled in front of it where a LOCK prefix would normally go. |
| 21:20 | < RichyB> | Then when you go multithreaded, pause everything, replace the NOP sleds with LOCK prefixes and then go on. |
| 21:39 | | Turaiel[Offline] is now known as Turaiel |
| 21:40 | | * ToxicFrog|W`rkn unhappily pokes ZFS with a stick |
| 21:46 | <&ToxicFrog|W`rkn> | I was hoping that I could have a single z-pool with some datasets fully replicated and others using RAID-Z, but apparently this isn't possible |
| 21:50 | <@Azash> | ToxicFrog|W`rkn: I have a friend who's pretty into zfs if you want me to ask? |
| 21:51 | <&ToxicFrog|W`rkn> | Azash: sure, but the documentation I'm looking at seems pretty clear on the topic :( |
| 21:52 | <&ToxicFrog|W`rkn> | (I was hoping I could have a fully replicated /home and raidz /storage in the same z-pool, as that saves me having to pre-allocate space for different filesystems, which is a massive headache and part of what I was hoping to get away from with ZFS) |
| 21:53 | <&ToxicFrog|W`rkn> | I'm also running into other places where ZFS seems to be way less flexible than mdadm, like you can't add/remove disks from a raid-z |
| 21:54 | <@Azash> | Ah well |
| 22:02 | | VirusHome is now known as Pandemic |
| 22:02 | | mode/#code [+o Pandemic] by ChanServ |
| 22:17 | < RichyB> | wtf numpy |
| 22:18 | < RichyB> | I only asked you to allocate 1<<30 int32_ts. |
| 22:18 | < RichyB> | Oh wait, maybe I accidentally took too many copies and exceeded RAM. |
| 22:20 | <&ToxicFrog|W`rkn> | Ok, it looks like there's actually a per-dataset option, copies, which requests some amount of replication on top ofwhatever is provided by the pool itself |
| 22:20 | <&ToxicFrog|W`rkn> | You can't ask for full mirroring but you can ask for up to three copies |
| 22:21 | | Vornicus [vorn@ServerAdministrator.Nightstar.Net] has joined #code |
| 22:21 | | mode/#code [+qo Vornicus Vornicus] by ChanServ |
| 22:21 | <&ToxicFrog|W`rkn> | So I could in fact have /home with replication and /storage with just parity in the same z-pool |
| 22:21 | < RichyB> | ToxicFrog|W`rkn: have you read the "Getting started with ZFS" document from the FreeBSD project? |
| 22:22 | <&ToxicFrog|W`rkn> | Is this the "zfsquickstartguide" on the wiki or something else? |
| 22:22 | < RichyB> | Yes. |
| 22:22 | < RichyB> | It might all just be stuff that you already know, but I do recommend skimming it for a brief overview of the kinds of things that you can do without having to go spelunking into every single tool's man page. |
| 22:23 | <&ToxicFrog|W`rkn> | Done. |
| 22:23 | <&ToxicFrog|W`rkn> | And yeah, everything there i already knew (although copies I only just discovered) except for periodic.conf, which I think is BSD specific? |
| 22:25 | < RichyB> | periodic.conf sounds like it's prolly FreeBSD-specific. |
| 22:25 | < RichyB> | I do not entirely understand the appeal of things like that. |
| 22:25 | < RichyB> | We already *have* crond, why for add more mechanisms for running things routinely? |
| 22:26 | < RichyB> | Oh, actually that one's pretty sensible. |
| 22:27 | <&ToxicFrog|W`rkn> | Oh? |
| 22:27 | | ToxicFrog|W`rkn is now known as ToxicFrog |
| 22:28 | <&ToxicFrog> | Anyways, it looks like the main decision for me here is between the ease of changing the number of drives in mdadm (admittedly not actually that easy, but it is at least possible) and the delicious features of zfs. |
| 22:28 | < RichyB> | Yeah, it's a collection of commonly-desirable cron jobs, not another damn. |
| 22:28 | < RichyB> | *not another damn pointless cron replacement |
| 22:28 | <&ToxicFrog> | Aah. So you set what tasks you want and when, and it handles the details of telling cron about them. |
| 22:29 | < RichyB> | It's a bunch of default lines like "1 3 * * * root periodic daily" in /etc/crontab. |
| 22:31 | < RichyB> | Rather than being another thing like cron.daily and cron.weekly for dropping shell scripts into, it's a single script that reads a config file and possibly does various maintenance tasks that're appropriate for the base system. |
| 22:31 | < RichyB> | (I can see the appeal of /etc/cron.daily + friends but they make me irrationally angry.) |
| 22:39 | < RichyB> | oh my egads, matplotlib is seriously sexy. |
| 22:43 | | Tamber [tamber@furryhelix.co.uk] has joined #code |
| 22:43 | | mode/#code [+o Tamber] by ChanServ |
| 23:21 | | himi [fow035@0C0840.B22E58.E3471A.E028A1] has quit [Ping timeout: 121 seconds] |
| --- Log closed Thu May 09 00:00:31 2013 |