WARNING - some people have found that this method renders their machine unbootable, see discussion in comments. Use at your own risk!

Here how I managed to change "Core Single Language" Windows 8.1 language from Russian to English. Thanks to @mako response.

I've also made the same steps for Windows 10 Single Language.

First of all go and get a language pack you need, links for 8.1 are available here http://social.technet.microsoft.com/Forums/en-US/ec4419df-ea0b-4da9-8750-063126debe97/language-pack-not-usable-message-in-windows-81?forum=w8itprogeneral

For Windows 10 you can get ISO with all language packs at http://www.youtube.com/watch?v=kzfuX_e_6iY, links are inside video description. Video itself is not related. You need to download ISO for your particular build, mine was build #10240 x64. You can get build number by running command prompt and looking in the first line for "Microsoft Windows [Version 10.0.10240]". I've made these steps after I upgraded from Windows 8.1 to Windows 10. As you know, if you changed your display language in 8 or 8.1 for single language version, then after upgrading to 10 you get your original display language, in my case it was Russian. I had to do these steps to set display language to English once again.

Now to the point. (package names/versions are different Windows 10)

Download a CAB file (language pack) for your language. Then rename the file to lp.cab and put it into C:\ drive folder.

Then run a command line (cmd, command prompt) as administrator and run this command there:

This will install new language pack into the system. But you can not change language yet. You need to uninstall your old language to apply new one. Run this command now in the same command prompt window:

This will give you a list of installed language packs, something like this:

You need to get a package identity for you current language which needs to be removed. I needed to remove Russian so I get "Microsoft-Windows-Client-LanguagePack-Package~31bf3856ad364e35~amd64~ru-RU~6.3.9600.16384" as a package name. ru-RU part is a language name. If you need to remove English than look for a package name with en-US part.

OK, now you need to uninstall unneeded language pack, run this command, change package name to one found earlier:

After that it will ask you to restart computer to apply changes, respond Y and after restart you will get a new system language. If you ever need to change it back just make the same procedure for different language pack.