Recently it was reported the Federal Office of Personnel Management was hacked by the Chinese in December of 2014. It is possible that around 4 million current and former federal employees data was compromised.(1) So how do we protect this data?
Maybe the better question is why do we store this data? If much of this data was not being stored online in databases, files and on a network, then even if the systems are compromised the data is not there to take. How many of these "former" employees of the Federal government have been gone for decades or longer? How long should their records be kept? More importantly how long should their records be kept "online"?
Two words "retention policies".
Oh and here is another important word "liability".
In today's world of storage getting cheaper everyday, everyone seems complacent to just store information forever. Who cares if it is 20 years old. We "may" need it someday.
It is fair to say there is some data that needs to be kept for long periods of time. There is even some data that should never be removed. For example medical records should be kept for the lifetime of that individual and then some. I doubt you want to delete pictures of your kids to save space. What about your family tree, videos of your wedding, music collection. This data needs to be online and easy to access at anytime as well as backed up. (PLEASE everyone backup your data!)
Retention policies need to be setup on all classes of data. From documents, to pictures, video, databases, logs, etc. When it comes to data, content is the key to how long it should be kept. There are laws on how long some data is to be kept. Beyond legal requirements policies should be set based on usefulness of the data. 7 years is a fairly typical number used to hold financial data for most individuals and businesses as an example.
Keeping data too long is a liability to the holder of that data. If that data is breached or improperly used, the company, government agency, or individual holding onto that data can be liable. Not to mention it is never fun to have to tell millions of customers that their private information is in the hands of hackers soon to be on the black market for thieves all around the world.
Here are some solutions to managing data without deleting it:
Setup a tiered system for storage
This means have multiple levels of storage like
- Online
- Near-line
- Archived
Let me explain these in broad technical terms.
Online storage is just what it says, the data is online and easily/quickly accessible. Excellent security should be in place to protect this data. Online storage has many tiers of it's own based on how fast that data is needed (performance). I will not get into the technicalities of tiered storage systems, just look up "tiered storage models". This is the most expensive tier as the data changes a lot, requires fast storage systems/connections, and must be backed up on a regular basis.
Near-line storage is not accessible online directly by users but is easily put online if needed. Typically this would be onsite but on systems that may not be directly connected to the servers or network. Data administrators would need to bring this data online when needed. The advantage to this is that data is significantly less susceptible to hacking or rouge employees. Automated processes as well as manual processes would follow retention policies to move data from online to near-line storage.
Archived storage is the lowest tier and would typically be stored offsite. This would be kept for legal reasons and would be the slowest to bring online as well as the most work. This data would likely be stored on media like tapes or optical that is not easily modified. This protects the integrity of the data. Archived data would be nearly impossible for hackers or rouge employees to compromise as physical access would be needed to get to this data. Archived storage is also the cheapest to maintain.
Every organization should have policies in place already to deal with data retention as well as tiers of storage to comply with these polices. Organizations that do have policies in place and follow them are much less likely to have data compromised and if they do the amount of damage is minimized. As well if data is compromised your organization can show reasonable efforts were made to secure the data. Hopefully this will play out in the organizations defense in their favor - verses negligence.
Another benefit is deniability. If there is data then there is evidence. If there is no data there is no evidence. Plausible deniability. Not saying that there "is" wrong doing but keeping all your data around is just asking for trouble. Organizations must remember to follow the laws on keeping data for the minimal amount of time. Beyond that get rid of that data if you have no justifiable reason to keep it.
Cost is the main reason to get rid of data. Keeping data is expensive. Users don't think of this nor do most executives or bureaucrats. Data has to be stored on storage systems. In large network environments these storage systems are much more expensive than a desktop hard drive. Large storage systems can easily cost more than a luxury car and go up from there. Next you need to back that data up. So now you need an entirely separate storage system (normally cheaper than the primary) to keep a copy of all that data. But wait there's more... When you backup data there is more than one copy. There are hundreds of copies with version history and deleted data too.
Finally there is the space, power, cooling, and the management of all the data systems and backups. There are countless hours of IT employees' time to keep everything up and running so users can keep making yet more data.
References
(1) http://www.msn.com/en-us/news/us/chinese-hackers-breach-federal-government%E2%80%99s-personnel-office/ar-BBkHFqx
Thursday, June 4, 2015
Sunday, May 24, 2015
Cloud Caching
Businesses and individuals rely a lot on Cloud services for both work and personal use. The number one issue with Cloud services, applications, data storage, or whatever category they fall in is connectivity. If you have no Internet access you basically have little to no functionality with Cloud services.
You can mitigate the issue with having redundant paths to the Internet. This is very simple to do with a firewall that supports 2 or more Internet connections.
Beyond Internet connectivity there is also the issue of ownership. If your data is hosted by a service; lets take for example Google Docs. Then all your data is somewhere in the ether and you have no physical possession of that data. If Google was hacked, went down, or even worse went away then what do you do?
For a truly better experience here is my though on an approach that would get a lot more companies to trust the Cloud and move more services there.
I suggest an appliance that resides onsite that "caches" all the data and keeps a working copy of the services code local. This would allow users that have network access to this appliance to keep working with or without access to the Internet.
But as you read this you're thinking, how would that work? Why would any Cloud service let you have a copy of their proprietary code?
Create a standard that is secure and will emulate any Cloud based service in a protected virtual machine.
So just think if:
- Google Docs
- Salesforce.com
- QuickBooks Online
- Zoho
- iCloud
- Dropbox
- Office 365 (online office)
- Even email applications such as Gmail, Outlook.com, iCloud, Yahoo
- and many more...
Were ALL FULLY FUNCTIONAL WITH NO INTERNET CONNECTION!
Obviously this is temporary as the email would not talk to the outside world and data would not sync outside your office. But your users could continue to work without any interruption and still have access to data in the event of an Internet outage or a service outage from the Cloud provider themselves. Once Internet connectivity returned everything would sync back up like an offline mode sync.
The next benefit is speed. With a Cloud Caching appliance the code could execute locally in that secure/hardened virtual machine so your users would not send any data across the Internet and wait for that to come back. The load would essentially be "offloaded" from the Cloud service providers to your local network.
I would accomplish this with:
- Open standards and open source code to run the virtual machines.
- Vendors can then supply hardware and support while running the core with open source. Then adding any proprietary software on-top for management purposes, load balancing, etc.
- Each Cloud service provider would upload a new virtual machine or update their current one when they release any new code.
- Cloud service providers would create APIs to sync data into each customers repository.
For high level management owner/operators can set polices on their appliance such as:
- Version control, so if an update comes out they can control when it deploys and also have fail back if there is an issue with the new code. The Cloud service provider can override these polices to prevent corruption with the data.
- Bandwidth control.
- Use applications local or remote when Internet connectivity is available.
- Switch to applications locally if latency or bandwidth usage is high (definable).
- Customize URLs for internal access or to just have external URLs redirect to the internal appliance(s) using DNS control/redirection.
The possibility of even outright purchasing a perpetual license for a Cloud service would be feasible to then host on-site without Internet connectivity. This could also be used for Cloud services that are end of life or where the service went out of business.
The idea of a standard and openness is competition to keep prices low and make it so a proprietary appliance is not needed for each service. ISPs can even implement appliances in neighborhoods to keep access local and fast!
For Cloud services to really capture more business from those that have not yet adopted, this may be the best way forward. Hopefully someone will figure this all out and bring the Cloud home!
Monday, April 13, 2015
Smarter Traffic Control - Part 2
As promised from part 1 of the smarter traffic control I want to discuss stop signs.
These non-tech octagonal, red objects that most of us really don't like but deal with anyways are a daily part of our lives. Do a Google search on stop signs around the world, they are basically the same everywhere. There is no mistaking them, yet there is a fundamental problem with them, actually a few.
1. They are not always needed.
What I mean by this is that they may be put in place just to attempt to slow down traffic. There have been many studies that show more stop signs will actually increase the mean speed on a street(1).
As an example to this, the residential street where I grew up and my parents sill live on never had any stop signs except for one major intersection. The "loud" neighbors demanded traffic calming solutions to slow down the speeders on the street. First they tried speed humps but the fire department complained that it slowed their trucks down too much. So out went the speed humps and in went 4 stop signs at each intersection along the street. Now we have more problems with speeding but the cops seem happy enough to give out tickets for not making complete stops.
They even went as far as to put two stop signs on a corner just so there was an even amount between the street above and our street. As far as could I read into the local laws, that is not even an intersection and the stop signs are not even enforceable! My parents live at the end of the street. So each and every day minutes of their lives as well as all those around them are wasted unnecessarily. That can add up to almost a quarter of a day over the course of a year.
2. They can do more harm than good.
If a stop sign is unnecessary or misplaced it can actually cause more accidents(2). This is because there is a complacency from drivers that approach the intersection. Each driver is trusting that the other drivers will stop and follow the traffic rules. Instead most drivers that stop tens if not hundreds of times a day will make their stop and continue, many times without even checking if anyone else stopped.
3. When they are needed people are tired of them.
As I stated above drivers become complacent and blindly continue through an intersection after a stop. When going through many stop signs on a street drivers become "numb" and are more likely to make a mistake. Whereas on a street that may not be that busy and an intersection say has a two way stops for lower traffic side streets. Those drivers are much more likely to pay attention.
What we should do.
- Local governance should create rules about when a stop sign is required and these rules should used to decide if a sign is warranted or not. Such as:
* How many accidents does the intersection have each year?
* Are their blind corners near the intersection?
* Is it an actual intersection where there is cross traffic?
* Is there a substantial enough amount of cross traffic to cause one direction or another to have to wait for an extended amount of time?
* Will it make the intersection safer?
* Is it just being requested to slow traffic? If so then the request should be denied for this reason alone.
- Use of more yield signs where traffic speeds are slower. This will keep cars from having to stop all the time while still alerting drivers to pay attention for cross traffic.
- More 2 way stops vs all way stops. When there is cross traffic that is significantly heavier than say a side street. Why stop 100% of the cars when you only need to stop 5% of the cars.
- More traffic studies before installing a stop sign. Just do a simple traffic study on an intersection to decide if a stop sign is really needed. If it is determine what type of stop. 2 way, all way, 1 way, whatever way.
- Don't let squeaky wheels influence traffic control decisions. Democracy when it comes to traffic control does not work. Statistics and engineering need to decide how to control traffic, not loud mouth residents.
Lets get rid of stop signs as much as we can. Just think of the global impact all the cars stopping and starting for millions of stop signs around the world has on our environment. Not to mention all the brake pads, transmissions, engines, and anything else that moves on your car. Sure auto mechanics, part shops, and part manufactures love stop signs, but you don't have to.
Last note:
This goes out to all law enforcement that like to sit there and give out tickets to people that don't make a complete stop at a stop sign (California rolling stop). There are much better things for you to do than to prey on your tax paying constituents that "broke the law" by not coming to a full stop. They are not hurting anything and all you do is make people dislike you (law enforcement in general)... Yes it is revenue but it is not making anyone safer. In my opinion it causes resentment and will make people disrespect stop signs and those that enforce them. Now that can be dangerous.
References
(1) https://www.dmgov.org/Departments/Engineering/PDF/FAQ13_STOP_Signs_to_Reduce_Speeding.pdf
(2) http://www.creators.com/opinion/john-stossel/tear-down-the-stop-signs.html
These non-tech octagonal, red objects that most of us really don't like but deal with anyways are a daily part of our lives. Do a Google search on stop signs around the world, they are basically the same everywhere. There is no mistaking them, yet there is a fundamental problem with them, actually a few.
1. They are not always needed.
What I mean by this is that they may be put in place just to attempt to slow down traffic. There have been many studies that show more stop signs will actually increase the mean speed on a street(1).
As an example to this, the residential street where I grew up and my parents sill live on never had any stop signs except for one major intersection. The "loud" neighbors demanded traffic calming solutions to slow down the speeders on the street. First they tried speed humps but the fire department complained that it slowed their trucks down too much. So out went the speed humps and in went 4 stop signs at each intersection along the street. Now we have more problems with speeding but the cops seem happy enough to give out tickets for not making complete stops.
They even went as far as to put two stop signs on a corner just so there was an even amount between the street above and our street. As far as could I read into the local laws, that is not even an intersection and the stop signs are not even enforceable! My parents live at the end of the street. So each and every day minutes of their lives as well as all those around them are wasted unnecessarily. That can add up to almost a quarter of a day over the course of a year.
2. They can do more harm than good.
If a stop sign is unnecessary or misplaced it can actually cause more accidents(2). This is because there is a complacency from drivers that approach the intersection. Each driver is trusting that the other drivers will stop and follow the traffic rules. Instead most drivers that stop tens if not hundreds of times a day will make their stop and continue, many times without even checking if anyone else stopped.
3. When they are needed people are tired of them.
As I stated above drivers become complacent and blindly continue through an intersection after a stop. When going through many stop signs on a street drivers become "numb" and are more likely to make a mistake. Whereas on a street that may not be that busy and an intersection say has a two way stops for lower traffic side streets. Those drivers are much more likely to pay attention.
What we should do.
- Local governance should create rules about when a stop sign is required and these rules should used to decide if a sign is warranted or not. Such as:
* How many accidents does the intersection have each year?
* Are their blind corners near the intersection?
* Is it an actual intersection where there is cross traffic?
* Is there a substantial enough amount of cross traffic to cause one direction or another to have to wait for an extended amount of time?
* Will it make the intersection safer?
* Is it just being requested to slow traffic? If so then the request should be denied for this reason alone.
- Use of more yield signs where traffic speeds are slower. This will keep cars from having to stop all the time while still alerting drivers to pay attention for cross traffic.
- More 2 way stops vs all way stops. When there is cross traffic that is significantly heavier than say a side street. Why stop 100% of the cars when you only need to stop 5% of the cars.
- More traffic studies before installing a stop sign. Just do a simple traffic study on an intersection to decide if a stop sign is really needed. If it is determine what type of stop. 2 way, all way, 1 way, whatever way.
- Don't let squeaky wheels influence traffic control decisions. Democracy when it comes to traffic control does not work. Statistics and engineering need to decide how to control traffic, not loud mouth residents.
Lets get rid of stop signs as much as we can. Just think of the global impact all the cars stopping and starting for millions of stop signs around the world has on our environment. Not to mention all the brake pads, transmissions, engines, and anything else that moves on your car. Sure auto mechanics, part shops, and part manufactures love stop signs, but you don't have to.
Last note:
This goes out to all law enforcement that like to sit there and give out tickets to people that don't make a complete stop at a stop sign (California rolling stop). There are much better things for you to do than to prey on your tax paying constituents that "broke the law" by not coming to a full stop. They are not hurting anything and all you do is make people dislike you (law enforcement in general)... Yes it is revenue but it is not making anyone safer. In my opinion it causes resentment and will make people disrespect stop signs and those that enforce them. Now that can be dangerous.
References
(1) https://www.dmgov.org/Departments/Engineering/PDF/FAQ13_STOP_Signs_to_Reduce_Speeding.pdf
(2) http://www.creators.com/opinion/john-stossel/tear-down-the-stop-signs.html
Smarter Traffic Control - Part 1
How many hundreds of hours in our lives have been wasted waiting at stop lights and stop signs!
Yes, they are necessary traffic control otherwise there would be chaos on the streets. But how many times have you said to yourself "there is no one else at this intersection, why do I need to stop". Or "the stupid light turned red and now the other light is green but no one is there".
All this stopping waste not only your time and millions of others but it waste gas, adds stress to our lives, as well as wear and tear on our cars (brakes, engine, transmission, etc). So what can be done to fix this problem while still having safe and effective traffic control?
Lets start with traffic lights. Traffic lights were around in the early 1900s but were manually operated and required a traffic operator to control them. Traffic lights today are computer controlled with inductive sensor loops in the street (you can see these most of the time) to sense if there is a car in that lane. The programming will then direct traffic accordingly and typically in the same pattern from one direction of the intersection to the next. Some traffic signals are smart enough to skip directions where there are no cars as well as adjust wait times based on traffic flow. One of the big problems with the induction loops is they sometimes do not sense motorcycles or small cars. As well as cars that have gone into the intersection too much (passed the line) or are not close enough to the line. To compensate for these issues many traffic signals will allow a direction to go even if it does not sense anything. This is why sometimes in the middle of the night when you are the only one at a light but the other direction is green.
So lets get smarter about this. MUCH SMARTER!
Put cameras on the intersections to watch what is actually in the lane. Not a red light camera, those are lame. I am talking a camera that can actually see things like a motorcycle (or not) as well as cars that are coming toward the intersection (or not). These cameras could be visible, infrared, as well as thermal or all of the above to allow the algorithms to clearly identify traffic. Now how will this help.
- Less waiting for non-existent cross traffic.
- Less stopping of cars that are almost to the intersection but have not tripped the inductive pickup.
- No more having to show a green light to nothing or no one.
- Trigger a light change before an approaching car even needs to stop.
Now for traffic flow there are lights that are designed to talk to other nearby intersections. But it seems like they are being used more for speed control than for traffic flow. I always felt that the purpose of traffic lights is to stop you as much as possible. The real purpose is to (or should be) to get as much traffic as possible through an intersection as safely as possible. With this goal in mind we need smarter code to look at traffic patterns including time of day, day of the week, is it a holiday, is there an accident or emergency nearby, density of traffic in the different directions, speed of traffic between intersections, and so on.
To have an algorithm that can control traffic signals you need a lot more input beyond inductive sensor loops in the street. Cameras are a good start but how about talking to Google maps or Waze for example. How about a calendar so the lights know it is a work holiday, or that it is a weekend. What about when a school gets out 2 miles away? These factors dramatically affect traffic and should be taken into consideration.
Now for action!
None of the above can be implemented for free so there needs to be a justification to cities and other governing jurisdictions to flip the bill for these changes. Plus much of this technology does not yet exist but if the demand was there I guarantee it would. Think of the benefits:
- Better traffic flow
* Would get more people though the area faster
* More cars can get in and out of businesses faster
- Less stress on the drivers
- Less pollution from less stopping and accelerating
- Less wear on your car
* Brakes
* Engine
* Transmission
- Happier tax payers everywhere!
In the future there will be autonomous (self-driving) cars that will take the human error out of driving. So will we need traffic control in the future once people stop driving? Maybe not, or at least not as much of it.
Still thinking about stop signs... Will blog about that at another time. That is not going to be a short story.
Yes, they are necessary traffic control otherwise there would be chaos on the streets. But how many times have you said to yourself "there is no one else at this intersection, why do I need to stop". Or "the stupid light turned red and now the other light is green but no one is there".
All this stopping waste not only your time and millions of others but it waste gas, adds stress to our lives, as well as wear and tear on our cars (brakes, engine, transmission, etc). So what can be done to fix this problem while still having safe and effective traffic control?
Lets start with traffic lights. Traffic lights were around in the early 1900s but were manually operated and required a traffic operator to control them. Traffic lights today are computer controlled with inductive sensor loops in the street (you can see these most of the time) to sense if there is a car in that lane. The programming will then direct traffic accordingly and typically in the same pattern from one direction of the intersection to the next. Some traffic signals are smart enough to skip directions where there are no cars as well as adjust wait times based on traffic flow. One of the big problems with the induction loops is they sometimes do not sense motorcycles or small cars. As well as cars that have gone into the intersection too much (passed the line) or are not close enough to the line. To compensate for these issues many traffic signals will allow a direction to go even if it does not sense anything. This is why sometimes in the middle of the night when you are the only one at a light but the other direction is green.
So lets get smarter about this. MUCH SMARTER!
Put cameras on the intersections to watch what is actually in the lane. Not a red light camera, those are lame. I am talking a camera that can actually see things like a motorcycle (or not) as well as cars that are coming toward the intersection (or not). These cameras could be visible, infrared, as well as thermal or all of the above to allow the algorithms to clearly identify traffic. Now how will this help.
- Less waiting for non-existent cross traffic.
- Less stopping of cars that are almost to the intersection but have not tripped the inductive pickup.
- No more having to show a green light to nothing or no one.
- Trigger a light change before an approaching car even needs to stop.
Now for traffic flow there are lights that are designed to talk to other nearby intersections. But it seems like they are being used more for speed control than for traffic flow. I always felt that the purpose of traffic lights is to stop you as much as possible. The real purpose is to (or should be) to get as much traffic as possible through an intersection as safely as possible. With this goal in mind we need smarter code to look at traffic patterns including time of day, day of the week, is it a holiday, is there an accident or emergency nearby, density of traffic in the different directions, speed of traffic between intersections, and so on.
To have an algorithm that can control traffic signals you need a lot more input beyond inductive sensor loops in the street. Cameras are a good start but how about talking to Google maps or Waze for example. How about a calendar so the lights know it is a work holiday, or that it is a weekend. What about when a school gets out 2 miles away? These factors dramatically affect traffic and should be taken into consideration.
Now for action!
None of the above can be implemented for free so there needs to be a justification to cities and other governing jurisdictions to flip the bill for these changes. Plus much of this technology does not yet exist but if the demand was there I guarantee it would. Think of the benefits:
- Better traffic flow
* Would get more people though the area faster
* More cars can get in and out of businesses faster
- Less stress on the drivers
- Less pollution from less stopping and accelerating
- Less wear on your car
* Brakes
* Engine
* Transmission
- Happier tax payers everywhere!
In the future there will be autonomous (self-driving) cars that will take the human error out of driving. So will we need traffic control in the future once people stop driving? Maybe not, or at least not as much of it.
Still thinking about stop signs... Will blog about that at another time. That is not going to be a short story.
Tuesday, April 7, 2015
Feel at Home Wherever your Office is
For many years I had an office or should I say a cubical until I finally became self employed. I would put up a few pictures have some toys to play with on my desk and a few other things that made my daily place of work feel a bit more like mine.
But office spaces are changing. Now there are people working from home, sharing desk, open areas with desk all facing each other, etc. These changes are making it difficult to have a personal space that feels like it is "your" desk.
Here is my suggestion. Create some simple cloud based software (why cloud, I will get to that in a minute) and then a simple adapter that will connect to TVs and monitors with very little to no configuration. Think HDMI, Display Port, DVI, VGA, USB. You can then have a personalized space where you can virtually put up pictures, toys, things to play with, notes, a bulletin board, etc. Think widgets galore that you can download and customize. If you have a touch screen even better! Or you could have a device that connects to the side of the screen that will track your finger to make it a touch screen (these already exist).
Now when you sit down and log in to your work computer a proximity device will activate your profile from the cloud and download all your widgets and settings. Including up to date pictures of your family, friends, dog, cat, what you did on your vacation, just save the inappropriate stuff for home. Your profile would even know what devices are where so that picture of you that really shouldn't exist but you think its funny anyways would only display at home.
Your work could have policies that might only allow approved widgets as well as being able to block some or all of your content if there was a complaint. If your work provided the hardware they could register that hardware under their account thus giving them control over it. The content that you own would not be able to be modified or downloaded but could be censored if there was a complaint. The owner/controller of the hardware would never be able to see much more than a small thumbnail representation of your virtual office screens for privacy. Your company would also likely set policies to limit sharing so that you don't waste all day sharing things on their equipment with your friends outside of work.
Based on where you are you can even share with other people. This can be a shared task list and would be based on your location. You could slide a note over to another co-worker or friend from your screen to theirs. The company itself could have a company wide calendar that is mandatory on all screens they own/control.
Customization does not have to stop with a few screens. It could play music (or share music with others that subscribe to you, think being a DJ at work), control the lighting, temperature, how about if your desk itself was a screen.
Now for the cloud part. Here are the reasons for being in the cloud.
1. Portability - Does not matter where you are, just need an Internet connection.
2. Ownership - Do you want your work "owning" or storing personal pictures or communications?
3. Apps or widgets - Almost like an app store but with widget plugins.
4. Move data around - Because everyone is connected you could use this to send files to each other. Would be limited to just data files, no executable files.
Now to execute this I would suggest a simple Android based system since that eco-system is already developed and is open-sourced. There are many Android devices that simply plug right into an HDMI port (as an example) that are ready to go. Just need some software and a catchy name...
Wednesday, April 1, 2015
3D Printers Go Mainstream
In the next 5-10 years I think it will be common place to have 3D printers in the home and to have more advanced 3D printers at businesses printing everything from trinkets to replacement parts.
Instead of having to stock thousands of parts an auto parts store could simply have some 3D printers that can print parts containing plastic all the way to tempered metals. Even as advanced as printing circuit boards. In a home a simple 3D printer that can product plastic pieces for toys, trinkets, parts, etc.
The uses are endless, here are a few examples I came up with:
- Installing a new appliance, instead of including 5 different parts depending on how you install the appliance it could simply come with QR codes to scan where you can download the exact parts you need and print them. You could even customize the color and size of the part. This would reduce waste and make the install more precise.
- Working on your car and you need a special retainer clip. This has happened to me and they are cheap but I have to drive down to the auto parts store to get one. What if I could go online and print a few in minutes. No wasted time or gas.
- Toy broke and it just needs an arm. In the future toys should have a code to scan on them where you can print replacement parts online. Just go to the manufactures website and put in the code from the toy. an hour or so later you have your replacement part and the toy is as good as new.
- 3D printers might even be able to repair cracked or broken items. Just place an item in a printer that can scan the item and patch or fill in the problem areas.
- Custom create ergonomic mice, handles, grips, inserts, and countless other custom items that fit your hands, feet, whatever on your body. Just imagine taking a few pictures or a 3D scan of your hand to print a customized mouse body that perfectly fits your hand. A vendor could sell the internal pieces of a mouse, send you to their website where you can upload a 3D scan of your hand and in return download a 3D model to print the perfect mouse for you.
This technology is rapidly evolving and will soon be mainstream. One of the main issues with 3D printing is what you print. You need a 3D model to print from and either you have to make it yourself or you have to download it. There needs to be money involved otherwise people are not going to spend their time making these 3D models for download.
I am not the biggest fan or DRM (Digital Rights Management) but in this respects I think there needs to be some sort of DRM like system that allows someone to create a 3D model, sell it, and then protect its usage.
Lets say for example you want to print a cool new toy for your kid and there is a 3D model for $39 online. You go online, pay for the model and print it. A few hours later your kid has a new toy. But what is to prevent you from making 40 of these or a 100? What is to prevent you from sharing this with your friends?
I suggest a DRM type platform where the printer itself has a standard it follows to only allow the product to be printed how ever many times you paid for it if there is DRM on the model. The seller would embed a serial number for that download and the printer would redeem that serial number when the print is successful.
Obviously with any DRM they are never 100% secure .But it would keep honest people honest and those that spend a lot of time developing the toys, parts, etc would be much more likely to be paid.
3D printing is already a revolution that is in it's infancy. Mark my words it will be as common as having a microwave or a TV. Everyone will have a 3D printer someday and there will be less and less trips to the store for parts and pieces.
Sunday, December 14, 2014
Connection Monitoring Software
My last post "Connection Quality" went into technical detail about how a slow Internet connection can be troubleshot to determine the cause of the slowness.
So here is my idea on a software program that will work with any application that uses the Internet for connectivity such as streaming multimedia, remote connectivity such as Citrix, VMware View, RDP, and online gaming to name a few.
Lets start with an example of a Citrix session using the ICA protocol. A student is in a coffee shop and connects to the schools Citrix farm where they have access to an online portal with different apps for their classes. The student has very poor performance and calls the help desk. After some troubleshooting over the phone the help desk determines that it is the Internet connection at the coffee shop.
Now lets introduce the concept of software the not only monitors the quality of the connection but will also pinpoint where the issue is.
A small piece of executable would run as a service or in the background all the time. This would sense when a network connection is open that the software has the ability to monitor. It would then present a small graph or meter overlay on the screen somewhere not in the way. This would only show when the connection first starts or when the quality is bad or changes. These display settings would all be configurable including the thresholds to what is a "bad" or "good" connection.
Because the Citrix connection is connecting to a server at the schools data center we need to monitor the entire path. This means pings, traces, quality of service, throughput on the protocols being used, dropped packets etc all along the route. The software may identify that the issue is with the router or Internet connection itself at the coffee shop. The software is smart enough to not only test the speed and latency from inside the coffee shop but also ask an external server to test the path from outside the firewall. The results are then compared and it verifies that in this instance it is indeed a slow connection at the coffee shop.
The user is then notified that they should use a different Internet connection otherwise they may have a very poor experience with their Citrix connection.
Now lets say the issue was downstream with another router. The software would identify that and alert the user that there is an issue along the path and they may notice a slow connection. It will suggest trying another Internet connection. If the route is nearby possibly allowing them through a proxy (likely a premium service that the school might provide or the user can pay for) that will take a different path and reroute their traffic around that problem router.
All of this information is helping the user, reducing help desk calls and more importantly is reporting anonymously back to the central servers what routers are causing issues. ISPs all over the world can subscribe for free to this database for alerts. So when a router they are responsible for is an issue they will be notified. Thus improving the experience for everyone around the world.
It is even feasible that the central servers when they see a large amount of issues on particular routes can automate signals to subscribing routers to "poison" a route making routers go around that route instead of continuing to be a problem.
A website that would be accessible to the general public would graphically represent issues around the world as well as general statistics to what sites are busy, what games people are playing the most, etc, etc.
The primary revenue stream for this product would be that the software itself is paid for by the end connection point owners. In this scenario the school. The end users would benefit from being notified of a bad connection and the connection point owner would have reduced help desk calls to troubleshoot bad connectivity. Additional revenue streams such as a proxy to go around troubled routers and provide caching services could be paid for by the end user. In this case the student. If the school wanted to provide this service they could provide it only when connecting to their services.
I would suggest a free version that offers basic monitoring of the connection quality but does not get down to the protocol level and will only monitor out to a few hops. This will keep check on the quality of the local Internet connection. If they connect to a resource that pays for the service it will automatically engage the full feature set of quality monitoring and proxy services if that is also paid for.
So here is my idea on a software program that will work with any application that uses the Internet for connectivity such as streaming multimedia, remote connectivity such as Citrix, VMware View, RDP, and online gaming to name a few.
Lets start with an example of a Citrix session using the ICA protocol. A student is in a coffee shop and connects to the schools Citrix farm where they have access to an online portal with different apps for their classes. The student has very poor performance and calls the help desk. After some troubleshooting over the phone the help desk determines that it is the Internet connection at the coffee shop.
Now lets introduce the concept of software the not only monitors the quality of the connection but will also pinpoint where the issue is.
A small piece of executable would run as a service or in the background all the time. This would sense when a network connection is open that the software has the ability to monitor. It would then present a small graph or meter overlay on the screen somewhere not in the way. This would only show when the connection first starts or when the quality is bad or changes. These display settings would all be configurable including the thresholds to what is a "bad" or "good" connection.
Because the Citrix connection is connecting to a server at the schools data center we need to monitor the entire path. This means pings, traces, quality of service, throughput on the protocols being used, dropped packets etc all along the route. The software may identify that the issue is with the router or Internet connection itself at the coffee shop. The software is smart enough to not only test the speed and latency from inside the coffee shop but also ask an external server to test the path from outside the firewall. The results are then compared and it verifies that in this instance it is indeed a slow connection at the coffee shop.
The user is then notified that they should use a different Internet connection otherwise they may have a very poor experience with their Citrix connection.
Now lets say the issue was downstream with another router. The software would identify that and alert the user that there is an issue along the path and they may notice a slow connection. It will suggest trying another Internet connection. If the route is nearby possibly allowing them through a proxy (likely a premium service that the school might provide or the user can pay for) that will take a different path and reroute their traffic around that problem router.
All of this information is helping the user, reducing help desk calls and more importantly is reporting anonymously back to the central servers what routers are causing issues. ISPs all over the world can subscribe for free to this database for alerts. So when a router they are responsible for is an issue they will be notified. Thus improving the experience for everyone around the world.
It is even feasible that the central servers when they see a large amount of issues on particular routes can automate signals to subscribing routers to "poison" a route making routers go around that route instead of continuing to be a problem.
A website that would be accessible to the general public would graphically represent issues around the world as well as general statistics to what sites are busy, what games people are playing the most, etc, etc.
The primary revenue stream for this product would be that the software itself is paid for by the end connection point owners. In this scenario the school. The end users would benefit from being notified of a bad connection and the connection point owner would have reduced help desk calls to troubleshoot bad connectivity. Additional revenue streams such as a proxy to go around troubled routers and provide caching services could be paid for by the end user. In this case the student. If the school wanted to provide this service they could provide it only when connecting to their services.
I would suggest a free version that offers basic monitoring of the connection quality but does not get down to the protocol level and will only monitor out to a few hops. This will keep check on the quality of the local Internet connection. If they connect to a resource that pays for the service it will automatically engage the full feature set of quality monitoring and proxy services if that is also paid for.
Tuesday, December 9, 2014
Connection Quality
The world today is so connected that a computer is almost useless without an Internet Connection. But many people do not have a good connection to the Internet. More specifically the route they have from their computer is to the backbones of the Internet is slow. But what is slow? A slow or poor Internet connection is based on many factors but ultimately it is the perception to the end user of having to wait. Here are some common issues that cause "slowness":
Latency: How long it takes for a single packet to make a round trip in milliseconds. Less is better, anywhere from less than 1ms for a local network to 200ms for an Internet connection. Typically you want this to be under 60ms for a "good" connection.
Jitter: The amount of delta from one ping to the next, generally this is an average. Less is better, under 20ms is good for most connections.
Packet Loss: How many packets are lost in transmission. Less is better, should be well under 1%.
Hops: How many routers a packet must travel through. Less is better. There is more chance of packet loss with more hops.
As you can see there are many factors involved in that little TCP packet that goes from your computer to the server for data and back. TCP is a reliable protocol whereas UDP (normally for streaming video and music) is an unreliable protocol. TCP will re-transmit data when it does not receive it. There is also a handshake from end to end to confirm that the data was received. Along with this is check-sums to prevent corrupt data or incomplete data being received.
All this extra payload requires some overhead. Most do not realize that there is upload bandwidth used when downloading and vise versa. So when your ISP says you have a 10 Mbps download and a 512k upload, it is possible that your download speed is actually hindered by the upload limitation. For those that want to understand the technicality of this here goes...
A typical packet size is 1500 bytes (jumbo packets are larger and we are not going to talk about that here). A header is typically 40-60 bytes. The header has information on where the packet come from, where it is going, and what is in the packet. If the header is 40 bytes then that leaves 1460 bytes for data. This means a minimum of 2.7% overhead. Now if you are not using all the space in the packet, lets say only 100 bytes of data is sent then the overhead would be 40%. If you are doing something like playing an online game that is sending lots of small pieces of data you could have a high overhead and now your Internet connection is saturated with lots of small packets talking back and forth.
Programmers are smart and are pretty efficient with the transmission of packets. So if there is a request of data of say your position in an online game, instead of sending 10 small packets the software might combine the data request into 1 larger packet. Thus less overhead and if the connection is bad it is faster to retry sending 1 packet vs 10.
SOOOOooooo what can you do to improve your connection speed. There are lots of tricks out there including changing your MTU (maximum transmission units or packet size), installing software that promises to make your connection faster, or simply get a better ISP. First of all the MTU size is not likely going to do much as most network equipment will automatically pick the best MTU. Software installed on your computer is generally a good way to get malware or junk software. The best solution is to get the best ISP you can.
Money is not everything, so paying for the fastest connection might be a waste of that money. When you pay for a connection and there are different tiers of speed all the ISP is doing is "capping" the speed based on what you pay. For example if a cable company will provide 10 Mbps down and 2 Mbps up (I will refer to as 10/2 going forward, or down/up), 25/5, and 50/10. These speeds are controlled by software. So when your connection approaches these set limits it will slow you down. This is also known as packet shaping. Also some connection have a "burst" where the first say 10 MB (that's mega bytes) of data will download twice as fast and then it will slow down to what you pay. This makes customers happier and makes speed test results much nicer too.
The problem is the connection quality is the same, you just have a speed limit. Think of it as driving around a fast car but your not allowed over 100 mph or 80 mph. Do you really need to go that fast? If you check your email and surf the web then you need a basic speed connection. If you stream movies all day and multiple at a time then you need the faster speed. For gaming it really does not matter as the bandwidth is generally not that high. Latency is a more important factor.
How can you check quality of your connection? Here are a few sites that I use that will help.
http://www.pingtest.net/ (requires Flash and Java)
http://www.speedtest.net/ (requires Flash)
http://speedof.me/ (requires an HTML 5 browser like Chrome, Firefox, or IE 10+)
But these test are not showing the whole picture. They are testing your connection from your computer to their servers. What if the problem is not your connection. What if the issue is a router between you and your destination? A few quick troubleshooting test will help with this. Here is what I do to check my connection.
1. Go to several large websites to see if they all load slow. Like Google, Yahoo, MSN, Ebay, etc. If they all load slow then it is likely that the issue is with your computer or your connection.
2. Try another computer on the same network. Again if the same issue occurs then it is likely your connection.
3. Start a trace. In Windows open a command prompt (go to the run dialog and type CMD then press enter). Use the command tracert and then an address like this (without quotes) "tracert google.com". You will see a list of addresses and some will have names next to them. These are the "hops" or routers that your connection is sending packets through to get to Google.com. If you see an * meaning no reply don't worry as some routers will not reply, that is normal. Look down this list, if you see very high times in ms like well over 100 then that is likely where the slow down is. If those high numbers are at the beginning of the list the issue is close to you. If the numbers are near the bottom the issue is more toward Google's end.
4. Path Ping is another command that is good for troubleshooting. It is like a Trace Route but will do more analysis over the path with lots of pings. This takes longer to run but can provide more detailed information on where there is an issue.
5. Reset your network equipment as that generally never hurts. Do some of these test again to see if the problem is still there. If it is then you might have to contact your ISP. If the issue is further downstream then sorry there is nothing you can do about it. Just wait and try again later.
Each time you make a connection the path may change. Routers are constantly talking to each other to find the best route. So if one router goes down or a link is really busy they may all send out signals to find a better path. This could change the route and will "fix" that problem you had. Go ahead and try it. Do the same trace every day and at different times. You may find that the path changes a bit all the time. As ISPs add more bandwidth and more routers a better path may show up and the routers you talk to may send you on your way.
In conclusion the Internet is truly an amazing beast with massive redundancy everywhere. To take down the entire Internet is just about impossible and has been tried. Several attempts have been made to do this by targeting DNS servers around the world. This has caused slowness but never actually totally made the Internet inaccessible. DNS is what resolves a name like google.com to an address like 173.194.33.165. Think of directions to a house. If you know the persons name but have no address you will never find them.
So the next time you have slow Internet take a few minutes to understand what the issue is. Almost always it is going to be at end or with your equipment. Before you call tech support or freak out. Reboot your router, modem, and computer, basically reboot everything.
Friday, August 29, 2014
Thermal Cameras Everywhere... Beware...
Up until recently thermal cameras have been quite expensive and were generally only used in high end test equipment or military applications.
That is now changing with an announcement from Seek Thermal (1) where they will soon be offering a smartphone attachment for around $250 that will allow you to see temperature in real time on your smartphone screen.
The applications are endless! Just a few examples are:
Finding people or animals in the dark.
Checking the temp on the playground slide for you kids.
Looking for electrical shorts in a wall.
Looking for leaks in an air duct.
Testing the effectiveness of insulation.
Watching for forest fires.
And so much more...
Now as with all technologies people will always find a dark side. There are endless possibilities of abuse as well. Since this technology used to be difficult to obtain for most people it was not much of a concern. But with the significant lower price point more people will have access and thus more people will do bad things. This is just inevitable.
Here are a few ideas of malicious things you could do with a thermal camera:
Spy on people in the dark.
Look for a guard dog in a backyard to avoid.
Find a car in a parking lot that just parked so you know they are not likely to be back for a while.
Just use your imagination and think of the privacy implications.
Finally my last thought on this topic is that you may soon be able to tell what mood someone is in just by taking a thermal image of them. I belive it will not be long before there is an app that with a thermal sensor and possibly an IR sensor a user will be able to tell what mood a person is in. This could potentially change everything.
Let's say you want to ask the boss for money to find a project you're working on. Better check their mood first. Good, let's ask for money. Bad, let's wait. Very happy, let's ask for a lot of money.
I think this tech could also find its way into the dating scene. Imagine being able to read the body of the opposite sex. What they are feeling just by their thermal signature. We may have to wear thermal clothes just to protect ourselves from the curious or worse malicious.
References:
1. http://m.us.wsj.com/articles/smartphone-add-ons-offer-thermal-imaging-1408396425?mobile=y
Wednesday, August 6, 2014
Who is Responsible for an Algorithms Outcome?
I was reading an article today that stated:
"A Hong Kong court today ruled that local businessman Albert Yeung Sau-sing can sue the company over its Autocomplete function"(1)
Google states:
"...that it couldn't be held responsible for the suggestions made by Autocomplete. It was, it argued, a “mere passive facilitator”, with its algorithm based on the content of previous searches."(1)
This is very troubling to me as I understand that even though Google originally created the intelligence that makes the Autocomplete function. The data that it provides is not from any Google employee(s) but rather users of their products.
So it begs the question. Who is responsible for the outcomes of algorithms and eventually as these algorithms grow, artificial intelligence - "AI"?
Take the human example. You have a child that you raised and taught right from wrong to the best of your ability. One day they say something that you never told them and it gets them in a lot of trouble. Maybe they heard it from a friend, or TV, or even a song. Regardless, is the parent responsible for them being in trouble? And I don't mean responsible in the sense of the child possibly being a minor, I mean responsible in the sense that they somehow made that child say the inappropriate comment.
So then take Google's Autocomplete. I love it and it saves me a lot of typing. The algorithm behind it is amazing and I thank Google for this wonderful technology. But is it a stretch to say that when someone types their name and a negative or even liable phrase is suggested by the algorithm is somehow the fault of the algorithm itself? Results are only as good as the data. Plus lets be honest, sometimes the truth hurts.
To take the side of the Hong Kong court and force Google to modify their algorithm and have an exception list would be ludicrous. Not only is it blatant censorship but to police the Autocomplete suggestions and deal with removal request would be a huge drain on Google's resources.
This does not just have to do with Google. This is a much broader notion and will be a larger concern as more intelligence is integrated into our daily lives with algorithms. Ultimately we will (may already) have AI making decisions all the time that will affect us, possibly adversely.
As with a child that you teach right from wrong, they are going to do right sometimes and wrong others. Your goal as a parent is to give them tools so that they can make the best decisions for themselves.
Is this not what an intelligent algorithm is intended to do? Don't we want our computer creations to be smart enough to gather data available to them to make the best decision? Can Google or other software companies really force an algorithm to be nice all the time and be perfect?
I say no, because humans are not perfect therefore our creations cannot be perfect. But should censorship and the ignorance of a few destroy these amazing pieces of code that make our world in my option a much better place?
References:
(1) http://www.forbes.com/sites/emmawoollacott/2014/08/06/more-privacy-woes-for-google-this-time-its-autocomplete/
"A Hong Kong court today ruled that local businessman Albert Yeung Sau-sing can sue the company over its Autocomplete function"(1)
Google states:
"...that it couldn't be held responsible for the suggestions made by Autocomplete. It was, it argued, a “mere passive facilitator”, with its algorithm based on the content of previous searches."(1)
This is very troubling to me as I understand that even though Google originally created the intelligence that makes the Autocomplete function. The data that it provides is not from any Google employee(s) but rather users of their products.
So it begs the question. Who is responsible for the outcomes of algorithms and eventually as these algorithms grow, artificial intelligence - "AI"?
Take the human example. You have a child that you raised and taught right from wrong to the best of your ability. One day they say something that you never told them and it gets them in a lot of trouble. Maybe they heard it from a friend, or TV, or even a song. Regardless, is the parent responsible for them being in trouble? And I don't mean responsible in the sense of the child possibly being a minor, I mean responsible in the sense that they somehow made that child say the inappropriate comment.
So then take Google's Autocomplete. I love it and it saves me a lot of typing. The algorithm behind it is amazing and I thank Google for this wonderful technology. But is it a stretch to say that when someone types their name and a negative or even liable phrase is suggested by the algorithm is somehow the fault of the algorithm itself? Results are only as good as the data. Plus lets be honest, sometimes the truth hurts.
To take the side of the Hong Kong court and force Google to modify their algorithm and have an exception list would be ludicrous. Not only is it blatant censorship but to police the Autocomplete suggestions and deal with removal request would be a huge drain on Google's resources.
This does not just have to do with Google. This is a much broader notion and will be a larger concern as more intelligence is integrated into our daily lives with algorithms. Ultimately we will (may already) have AI making decisions all the time that will affect us, possibly adversely.
As with a child that you teach right from wrong, they are going to do right sometimes and wrong others. Your goal as a parent is to give them tools so that they can make the best decisions for themselves.
Is this not what an intelligent algorithm is intended to do? Don't we want our computer creations to be smart enough to gather data available to them to make the best decision? Can Google or other software companies really force an algorithm to be nice all the time and be perfect?
I say no, because humans are not perfect therefore our creations cannot be perfect. But should censorship and the ignorance of a few destroy these amazing pieces of code that make our world in my option a much better place?
References:
(1) http://www.forbes.com/sites/emmawoollacott/2014/08/06/more-privacy-woes-for-google-this-time-its-autocomplete/
Subscribe to:
Posts (Atom)