A much smarter money format?

impaktor
Posts: 1029
Joined: Fri Dec 20, 2013 9:54 am
Location: Tellus
Contact:

A much smarter money format?

Post by impaktor »

I've been going over the money format (old PR: #2774), and I think I have a working solution, at least on the C++ side, and I thought I'd present it here, since the discussion on github is spreading across multiple PRs. This is some of the improvements that the current one lacks:
  • Show minus sign correctly for negative amounts "-$10"
  • Have a thousand separator
  • Have an optional argument to remove fractional part
  • Allow decimal sign, thousand separator, and number of digits in a group to be configurable, by optional arguments
This should then be used on the Lua side by Format.Money(100), (to return $100.00) or Format.Money(100,false) (to return $100). Now I do wonder if this function, Format.Money, could call another Lua function, which makes use of the language resource to set decimal and thousand separator, since I really think this should be something dependent on the language chosen in pioneer, and not the locale of the computer.

I haven't thought this through fully, but perhaps something similar to:

Code: Select all

local l = Lang.GetResource("core")

Format.Money = function (money, show_fraction)
    local show_fraction_ = show_fraction or true;
    somefunction(money, show_fraction_, l.DECIMAL, l.THOUSAND_SEPERATOR, l.GROUPING)
end
Below is the code for a working program, to play around with (Note: requires C++11, tested on gcc4.7). Perhaps I should make it so that the fractional part is not removed if it's not ".00". Now I send an error/warning message, if the fractional part removed was non-zero.

Code: Select all

#include <iostream>
#include <iomanip>
#include <string>
#include <sstream>
#include <locale>

std::string format_money(long int money,
                         bool show_fraction=true, char decimal=',',
                         char thousand=' ', std::string group="\3")
{

  struct mymoney : std::moneypunct<char> {
  private:
    char decimal_point_;   // fraction delimiter symbol
    char thousand_sep_;    // seperator character for thousands
    int frac_digits_;      // where to put decimal sign
    std::string grouping_; // number of digits in a group

  public:
    mymoney(char decimal_point, char thousand_sep,
            std::string grouping, int frac_digits){
      this->decimal_point_ = decimal_point;
      this->thousand_sep_  = thousand_sep;
      this->frac_digits_   = frac_digits;
      this->grouping_      = grouping;
    }

    char do_decimal_point() const {
      return decimal_point_;
    }

    char do_thousands_sep() const {
      return thousand_sep_;
    }

    int do_frac_digits() const {
      return frac_digits_;
    }

    std::string do_grouping() const {
      return grouping_;
    }

    std::string do_curr_symbol() const {
      return "$";
    }

    std::string do_negative_sign() const {
      return "-";
    }

    pattern do_neg_format() const{
      return { {sign, symbol, value} };
    }
  };

  if(!show_fraction && money % 100)
    std::cerr << "Warning: Rounding off amount passed: " << money << std::endl;
  double long Money = show_fraction ? money : money*0.01;
  int frac_digits = show_fraction ? 2 : 0;

  std::locale loc(std::locale(), new mymoney(decimal, thousand,
                                             group, frac_digits));
  std::ostringstream oss;
  oss.imbue(loc);

  oss << std::showbase << std::put_money(Money);
  return oss.str();
}

int main(int argc, char **argv) {

  long double money = -130000012;
  std::cout << "number:\t" << money << std::endl;
  std::cout << "money:\t" << format_money(money) << std::endl;
  std::cout << "money:\t" << format_money(money,false) << std::endl;
  std::cout << "money:\t" << format_money(money,true, '#') << std::endl;

  return 0;
}
lwho
Posts: 72
Joined: Thu Jul 04, 2013 9:26 pm
Location: Germany

Re: A much smarter money format?

Post by lwho »

impaktor wrote:
  • Show minus sign correctly for negative amounts "-$10"
  • Have a thousand separator
  • Have an optional argument to remove fractional part
  • Allow decimal sign, thousand separator, and number of digits in a group to be configurable, by optional arguments
This should then be used on the Lua side by Format.Money(100), (to return $100.00) or Format.Money(100,false) (to return $100). Now I do wonder if this function, Format.Money, could call another Lua function, which makes use of the language resource to set decimal and thousand separator, since I really think this should be something dependent on the language chosen in pioneer, and not the locale of the computer.
I agree with all of this, but you are thinking to complex, how to achieve it. You don't need to go via a Lua functions for translated string, they are accessible from the C++ side:

In src/LangStrings.inc.h

Code: Select all

DECLARE_STRING(DECIMAL_POINT);
DECLARE_STRING(THOUSAND_SEPARATOR);
And where you need to use it:

Code: Select all

#include "Lang.h"
...
do_something_with(Lang::DECIMAL_POINT, Lang::THOUSAND_SEPERATOR);
SystemInfoView.cpp has some code, where you can peek how localized strings (with parameters) are normally output in our code base (e.g. in SystemInfoView::OnBodyViewed, SystemInfoView::UpdateEconomyTab, SystemInfoView::SystemChanged).
impaktor wrote:

Code: Select all

    somefunction(money, show_fraction_, l.DECIMAL, l.THOUSAND_SEPERATOR, l.GROUPING)
What is GROUPING for?
impaktor wrote: Perhaps I should make it so that the fractional part is not removed if it's not ".00". Now I send an error/warning message, if the fractional part removed was non-zero.
No, just round to the next integer. No need to spam warnings.
impaktor wrote:

Code: Select all

#include <iostream>
#include <iomanip>
#include <string>
#include <sstream>
#include <locale>
...
Maybe it's just me, but I just hate those stringstream based solutions (and I also prefer printf-like APIs over the stream API in most cases). I mean, this is a lot of boilerplate code doing nothing for something that you probably could code in a 10 lines function directly (and more readable). It's only putting two different customizable characters at certain places between digits. And even a naive solution will probably perform a lot better than this stream crap. (Yes, I know, that some C++ adepts will now tar and feather me). What are we using the locale API for when we are dictating the punctuation to use anyway?

Two open issues I'm wondering about
  • Locale is not equal language (e.g French: Canada, France, ...; Spanish: Spain, Americas, ...; English: Australia (cheers @robn ;)), UK, USA, South Africa,...). Is that only theoretical, or do we have languages where different locales with the same language have different number formats?
  • Are those "thousands" separators really thousands separators in each locale, or do some languages/locales group by another number of digits than three?
impaktor
Posts: 1029
Joined: Fri Dec 20, 2013 9:54 am
Location: Tellus
Contact:

Re: A much smarter money format?

Post by impaktor »

...they are accessible from the C++ side:
Awesome. Will look into it.
What is GROUPING for?
Grouping dictates if 10000 is printed as "10000" or "1,00,00" or "10,000".
...but I just hate those stringstream based solutions...


I thought printf was "C", and we should use stringstream as that is C++. I'm not sure how to use the locale library without using some form of C++ stream. Not using the locale library, would mean basically to re-implement it ourselves(?), as I see it, which defeats the purpose of having a locale library. Not sure I understood you correctly though.
...do we have languages where different locales with the same language have different number formats?


I hadn't thought of that, but sounds very unlikely to me. Perhaps an "old world perspective" of me, but if English, French, Portuguese, and Spanish have the same number format in all countries spoken, then might one assume this is not a problem?
Are those "thousands" separators really thousands separators in each locale, or do some languages/locales group by another number of digits than three?
wikipedia wrote:The groups created by the delimiters tend to follow the use of the local language, which varies. In European languages, large numbers are read in groups of thousands and the delimiter (which occurs every three digits when it is used) may be called a "thousands separator". In East Asian cultures, particularly China and Japan, large numbers are read in groups of myriads (10,000s) and the delimiter accordingly separates every four digits.[citation needed] The Indian numbering system is somewhat more complex: it groups the first three digits in a similar manner to European languages but then groups every two digits thereafter: 1.5 million would accordingly be written 15,00,000 and read as "15 lakh".
That page had a lot of fun info.
lwho
Posts: 72
Joined: Thu Jul 04, 2013 9:26 pm
Location: Germany

Re: A much smarter money format?

Post by lwho »

impaktor wrote: I thought printf was "C", and we should use stringstream as that is C++.
Yeah, that's what the C++ adepts will try to convince you of. But the following is perfectly valid (and ISO conforming) C++:

Code: Select all

#include <cstdio>
...
std::printf(...);
impaktor wrote: I'm not sure how to use the locale library without using some form of C++ stream. Not using the locale library, would mean basically to re-implement it ourselves(?), as I see it, which defeats the purpose of having a locale library. Not sure I understood you correctly though.
Well. if re-implement it ourselves, means 10 lines of easy to understand code, I would prefer that. What a locale library could buy us, is the the access to the locale database. But as I understand your code, you are going the other way around, i.e. telling it what to use, not asking it "what punctuation should I use for Martian German" ;).
impaktor wrote:I hadn't thought of that, but sounds very unlikely to me. Perhaps an "old world perspective" of me, but if English, French, Portuguese, and Spanish have the same number format in all countries spoken, then might one assume this is not a problem?
That's why I was asking, if anyone is aware of non-matching punctuation for different locales of the same language. If there aren't any, this doesn't need to bother us.
Are those "thousands" separators really thousands separators in each locale, or do some languages/locales group by another number of digits than three?
Ahh, that was what the GROUPING was for, right?
wikipedia wrote:The groups created by the delimiters tend to follow the use of the local language, which varies. In European languages, large numbers are read in groups of thousands and the delimiter (which occurs every three digits when it is used) may be called a "thousands separator". In East Asian cultures, particularly China and Japan, large numbers are read in groups of myriads (10,000s) and the delimiter accordingly separates every four digits.[citation needed] The Indian numbering system is somewhat more complex: it groups the first three digits in a similar manner to European languages but then groups every two digits thereafter: 1.5 million would accordingly be written 15,00,000 and read as "15 lakh".
Hah, caught you ;) You know that English is one of India's official languages? So, for English there are different rules. Though, I don't know, if I really would want to support such a mixed system. That's something were a locale library starts paying off.
lwho
Posts: 72
Joined: Thu Jul 04, 2013 9:26 pm
Location: Germany

Re: A much smarter money format?

Post by lwho »

Looking at the lists and map on Wikipedia, Spanish speaking countries seem to have different decimal "points": dot in Central America, comma in Spain and South America.
lwho
Posts: 72
Joined: Thu Jul 04, 2013 9:26 pm
Location: Germany

Re: A much smarter money format?

Post by lwho »

Maybe we should not over-complicate things. How's about not tying the number format to the language (strings) at all and let the user just select his thousands separator and decimal mark in the options.

I'm a bit unsure what to do about grouping. For every country, except for India, 3 digits seems to be one valid option (of possibly multiple options). But India has about 1/6th of the world population. So, not something one should just "drop under the carpet". On the other hand we don't have Hindi as a language, either. So, it seems not so many representatives in the Pioneer community.
impaktor
Posts: 1029
Joined: Fri Dec 20, 2013 9:54 am
Location: Tellus
Contact:

Re: A much smarter money format?

Post by impaktor »

I'm not at all sure I could write that in just ten lines of code. I suspect it would be at least the same amount of code as now. I'll think about it.
Ahh, that was what the GROUPING was for, right?
Correct. Then the thousand separator is typically ",", " ", or "´".

Also, I'm still thinking about what to do when the user is telling Format.Money to not show fractional part. This would be used for ship prices, and ship equipment (and possibly elsewhere). All ship prices should have zero fractional part, but say for some reason they don't. If we then round off the number then the amount of money withdrawn from players account will not be what is being shown on the screen, which feels like a bug. In such a case I think it would be better to actually just show the fractional part. We would immediately see that it looks ugly, and that the price hasn't been floored or similar.
impaktor
Posts: 1029
Joined: Fri Dec 20, 2013 9:54 am
Location: Tellus
Contact:

Re: A much smarter money format?

Post by impaktor »

How's about not tying the number format to the language (strings) at all and let the user just select his thousands separator and decimal mark in the options.
I'm still leaning towards having it, at least initially, connected to the language selected. For most languages it will be the right thing, I think. For some languages (like English_Indian) they will have to (for now) accept using UK/US number standard. Nothing stranger than UK and US having to cope with colour/color and all other quirks. Or all of us in non-former English colonies having to cope with "." being a fraction delimiter in all of computing/programming/internet. Either way, it will be an improvement to the current behaviour/behavior (:-P).

Below is a program one can play with, that does all of the things stated in my first post, but without using locale library or C++ streams. Note: I stole the do-while-loop from stackoverflow, and I don't know what license that will fall under.

Also, I don't fully understand the use of the n-variable, the way we/I use it below, all numbers (at that time) will always have two number fractional part, but is seems to do the right ting.

Code: Select all

#include <iostream>
#include <stdio.h>
#include <string>
#include <string.h>
#include <cmath>

std::string format_money(long int money, bool show_fraction=true,
                          char decimal_point=',', char seperator = ' ',
                          int group_digits=3){

  double Money = 0.01*double(money);
  const char *format = (Money < 0) ? "-$%.2f" : "$%.2f";

  char buf[64];
  snprintf(buf, sizeof(buf), format, std::abs(Money));

  char *pt;
  for(pt = buf; *pt && *pt != '.'; pt++) {} //Point pt at "."

  int n = buf + sizeof(buf) - pt;           //Length of fractional part
  int skipp = (Money < 0) ? 2 : 1;
  do{                                       //Step backwards, inserting spaces
    pt -= group_digits;                     //shift group_digits
    if(pt > buf+skipp){
      memmove(pt + 1, pt, n);               //allows overlap, unlike memcpy
      *pt = seperator;
      n += group_digits + 1;                //group digits + separator
    }else
      break;
  }while(true);

  char *pos = buf + strlen(buf) - 1;        //pos of last char stored in buf

  *(pos-2) = decimal_point;                 //change decimal delimiter symbol

  if(!show_fraction)
    if(*pos == '0' && *(pos-1) == '0')      //remove fractional part if zero
      *(pos-2) = '\0';

  return std::string(buf);
}


int main(int argc, char **argv) {

  long double money = 10;
  std::cout << "number:\t" << money << std::endl;
  std::cout << "money:\t" << format_money(money) << std::endl;
  std::cout << "money:\t" << format_money(money,false) << std::endl;
  std::cout << "money:\t" << format_money(money,true, '.', '\'') << std::endl;
  return 0;
}
lwho
Posts: 72
Joined: Thu Jul 04, 2013 9:26 pm
Location: Germany

Re: A much smarter money format?

Post by lwho »

impaktor wrote:I'm not at all sure I could write that in just ten lines of code.
Challenge taken ;)

I did not peek at your solution, yet. And only with a little bit of cheating (extracting the FillDigits function), it's 10 lines of function body.

Code: Select all

#include <cstdio>
#include <string>
#include <cmath>

namespace Lang {
    const std::string DECIMAL_MARK = ",";
    const std::string THOUSANDS_SEPARATOR = ".";
    unsigned NUMBER_GROUPING = 3;
}

inline std::string FillDigits(unsigned long long num, unsigned digits) {
    std::string result(std::to_string(num));
    return result.size() >= digits ? result : result.insert(0, digits - result.size(), '0');
}

std::string FormatMoney(double num, bool withCents) {
    const std::string prefix(num < 0 ? "-$" : "$");
    num = std::fabs(num);
    unsigned long long whole = withCents ? std::floor(num) : std::round(num);
    std::string result(withCents ? Lang::DECIMAL_MARK + FillDigits(static_cast<unsigned long long>(num * 100.0) % 100, 2) : "");
    if (whole == 0) return prefix + "0" + result;
    unsigned modulo = 10;
    for (int i = Lang::NUMBER_GROUPING; i > 1; modulo *= 10, --i);
    for (; whole >= modulo ; whole /= modulo)
        result.insert(0, Lang::THOUSANDS_SEPARATOR + FillDigits(whole % modulo, Lang::NUMBER_GROUPING));
    return prefix + std::to_string(whole) + result;
}

int main() {
    double values[] = { 0.0, 0.01, 0.49, 0.5, 999.49, 999.5, 1234.56, 70342.12, 999999.5, 5500100.42 };
    for (double x : values)
        std::printf("%.3f = %s = %s\n", x, FormatMoney(x, false).c_str(), FormatMoney(x, true).c_str());
    for (double x : values)
        std::printf("%.3f = %s = %s\n", -x, FormatMoney(-x, false).c_str(), FormatMoney(-x, true).c_str());
    return 0;
}
lwho
Posts: 72
Joined: Thu Jul 04, 2013 9:26 pm
Location: Germany

Re: A much smarter money format?

Post by lwho »

Funny, now I arrived at a C++-like solution (I only just discovered std::to_string() in C++11) and you at a C-like, though changing your includes to C++-conforming ones ("#include <stdio.h>" -> "#include <cstdio>", "#include <string.h>" -> "#include <ctring>" and putting std:: before the "C" functions yours would be valid C++ as well.

Code: Select all

#include <iostream>
#include <cstdio>
#include <string>
#include <cstring>
#include <cmath>
I also ignored India for now, as you probably noticed ;)

My code outputs "-$0" for something like -0.41 without cents. Though that was not really intended at first, I'm starting to consider this a feature rather than a bug. Being below zero is qualitatively different to above zero.

I'm starting to wonder, if "$" with localized numbers makes so much sense ;)

The code was developed in clean-room and is hereby licensed as GPLv3 ;)
Post Reply